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Some Notes 



Jean Gallier 



Abstract: These are notes on discrete mathematics for computer scientists. The presen- 
tation is somewhat unconventional. Indeed I begin with a discussion of the basic rules of 
mathematical reasoning and of the notion of proof formalized in a natural deduction system 
"a la Prawitz" . The rest of the material is more or less traditional but I emphasize partial 
functions more than usual (after all, programs may not terminate for all input) and I provide 
a fairly complete account of the basic concepts of graph theory. 
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Preface 



The curriculum of most undergraduate programs in computer science includes a course un- 
titled Discrete Mathematics. These days, given that many students who graduate with a 
degree in computer science end up with jobs where mathematical skills seem basically of no 
usejll one may ask why these students should take such a course. And if they do, what are 
the most basic notions that they should learn? 

As to the first question, I strongly believe that all computer science students should take 
such a course and I will try justifying this assertion below. 

The main reason is that, based on my experience of more than twenty five years of 
teaching, I have found that the majority of the students find it very difficult to present an 
argument in a rigorous fashion. The notion of a proof is something very fuzzy for most 
students and even the need for the rigorous justification of a claim is not so clear to most of 
them. Yet, they will all write complex computer programs and it seems rather crucial that 
they should understand the basic issues of program correctness. It also seems rather crucial 
that they should possess some basic mathematical skills to analyse, even in a crude way, 
the complexity of the programs they will write. Don Knuth has argued these points more 
eloquently that I can in his beautiful book. Concrete Mathematics, and I will not elaborate 
on this anymore. 

On a scholarly level, I will argue that some basic mathematical knowledge should be part 
of the scientific culture of any computer science student and more broadly, of any engineering 
student. 

Now, if we believe that computer science students should have some basic mathematical 
knowledge, what should it be? 

There no simple answer. Indeed, students with an interest in algorithms and complexity 
will need some discrete mathematics such as combinatorics and graph theory but students 
interested in computer graphics or computer vision will need some geometry and some contin- 
uous mathematics. Students interested in data bases will need to know some mathematical 
logic and students interested in computer architecture will need yet a different brand of 
mathematics. So, what's the common core? 

As I said earlier, most students have a very fuzzy idea of what a proof is. This is actually 
true of most people! The reason is simple: It is quite difficult to define precisely what a proof 

^In fact, some people would even argue that such skills constitute a handicap! 
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is. To do this, one has to define precisely what are the "rules of mathematical reasoning" 
and this is a lot harder than it looks. Of course, defining and analyzing the notion of proof 
is a major goal of mathematical logic. 

Having attempted some twenty years ago to "demystify" logic for computer scientists 
and being an incorrigible optimist, I still believe that there is great value in attempting 
to teach people the basic principles of mathematical reasoning in a precise but not overly 
formal manner. In these notes, I define the notion of proof as a certain kind of tree whose 
inner nodes respect certain proof rules presented in the style of a natural deduction system 
"a la Prawitz". Of course, this has been done before (for example, in van Dalen |32]) but 
our presentation has more of a "computer science" fiavor which should make it more easily 
digestible by our intended audience. Using such a proof system, it is easy to describe very 
clearly what is a proof by contradiction and to introduce the subtle notion of "constructive 
proof" . We even question the "supremacy" of classical logic, making our students aware of 
the fact that there isn't just one logic, but different systems of logic, which often comes as 
a shock to them. 

Having provided a firm foundation for the notion of proof, we proceed with a quick and 
informal review of the first seven axioms of Zermelo-Frankel set theory. Students are usually 
surprised to hear that axioms are needed to ensure such a thing as the existence of the 
union of two sets and I respond by stressing that one should always keep a healthy dose of 
skepticism in life! 

What next? Again, my experience has been that most students do not have a clear 
idea of what a function is, even less of a partial function. Yet, computer programs may 
not terminate for all input, so the notion of partial function is crucial. Thus, we define 
carefully relations, functions and partial functions and investigate some of their properties 
(being injective, surjective, bijective). 

One of the major stumbling blocks for students is the notion of proof by induction and its 
cousin, the definition of functions by recursion. We spend quite a bit of time clarifying these 
concepts and we give a proof of the validity of the induction principle from the fact that the 
natural numbers are well-ordered. We also discuss the pigeonhole principle and some basic 
facts about equinumerosity, without introducing cardinal numbers. 

We introduce some elementary concepts of combinatorics in terms of counting problems. 
We introduce the binomial and multinomial coefficients and study some of their properties 
and we conclude with the Inclusion-Exclusion Principle. 

Next, we introduce partial orders, well-founded sets and complete induction. This way, 
students become aware of the fact that the induction principle applies to sets with an ordering 
far more complex that the ordering on the natural numbers. As an application, we prove 
the unique prime factorization in Z and discuss CCD's. 

Another extremely important concept is that of an equivalence relation and the related 
notion of a partition. 
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We have included some material on lattices, Tarski's fixed point Theorem, distributive 
lattices, boolean algebras and Heyting algebras. These topics are somewhat more advanced 
and can be omitted from the "core" . 

The last topic that wc consider crucial is graph theory. We give a fairly complete pre- 
sentation of the basic concepts of graph theory: directed and undirected graphs, paths, 
cycles, spanning trees, cocycles, cotrees, flows and tensions, Eulerian and Hamiltonian cy- 
cles, matchings, coverings, and planar graphs. We also discuss the network flow problem and 
prove the Max-Flow Min-Cut Theorem in an original way due to M. Sakarovitch. 

These notes grew out of lectures I gave in 2005 while teaching CSE260. There is more 
material than can be covered in one semester and some choices have to made as to what to 
omit. Unfortunately, when I taught this course, I was unable to cover any graph theory. I 
also did not cover lattices and boolean algebras. 

My unconventional approach of starting with logic may not work for everybody, as some 
individuals find such material too abstract. It is possible to skip the chapter on logic and 
proceed directly with sets functions, etc. I admit that I have raised the bar perhaps higher 
than the average compared to other books on discrete maths. However, my experience when 
teaching CSE260 was that 70% of the students enjoyed the logic material, as it reminded 
them of programming. I hope that these notes will inspire and will be useful to motivated 
students. 
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Chapter 1 



Mathematical Reasoning, Proof 
Principles and Logic 

1.1 Introduction 

Mathematicians write proof; most of us write proofs. This leads to the question: Which 
principles of reasoning do we use when we write proofs? 

The goal of this Chapter is to try answering this question. We do so by formalizing 
the basic rules of reasoning that we use, most of the time unconsciously, in a certain kind 
of formalism known as a natural deduction system. We give a (very) quick introduction to 
mathematical logic, with a very deliberate proof-theoretic bent, that is, neglecting almost 
completely all semantic notions, except at a very intuitive level. We still feel that this 
approach is fruitful because the mechanical and rules-of-the-game flavor of proof systems 
is much more easily grasped than semantic concepts. In this approach, we follow Peter 
Andrew's motto [1]: 

"To truth through proof" . 

We present various natural deduction systems due to Prawitz and Gentzen (in more 
modern notation), both in their intuitionistic and classical version. The adoption of natural 
deduction systems as proof systems makes it easy to question the validity of some of the 
inference rules, such as the principle of proof by contradiction. In brief, we try to explain to 
our readers the difference between constructive and classical (i.e., not necessarily construc- 
tive) proofs. In this respect, we plant the seed that there is a deep relationship between 
constructive proofs and the notion of computation (the "Curry-Howard isomorphism" or 
"formulae-as-types principle", see Section [L71) . 
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1.2 Inference Rules, Deductions, The Proof Systems 



In this section, we review some basic proof principles and attempt to clarify, at least infor- 
mally, what constitutes a mathematical proof. 

In order to define the notion of proof rigorously, we would have to define a formal language 
in which to express statements very precisely and we would have to set up a proof system 
in terms of axioms and proof rules (also called inference rules). We will not go into this; 
this would take too much time and besides, this belongs to a logic course, which is not what 
CSE260 is! Instead, we will content ourselves with an intuitive idea of what a statement is 
and focus on stating as precisely as possible the rules of logic that are used in constructing 
proofs. Readers who really want to see a thorough (and rigorous) introduction to logic are 
referred to Gallier [IBj van Dalen [42j or Huth and Ryan [30j, a nice text with a Computer 
Science flavor. A beautiful exposition of logic (from a proof-theoretic point of view) is also 
given in Troelstra and Schwichtenberg [H], but at a more advanced level. You should also 
be aware of CSE482, a very exciting course about logic and its applications in Computer 
Science. By the way, my book has been out of print for some time but you can get it free 
(as pdf files) from my logic web site 

http: / /www. cis.upenn.edu/~jean/gbooks/logic. html 

In mathematics, we prove statements. Statements may be atomic or compound, that 
is, built up from simpler statements using logical connectives, such as, implication (if-then), 
conjunction (and), disjunction (or), negation (not) and (existential or universal) quantifiers . 

As examples of atomic statements, we have: 

1. "a student is eager to learn". 

2. "a students wants an A". 

3. "an odd integer is never 0" 

4. "the product of two odd integers is odd" 

Atomic statements may also contain "variables" (standing for abitrary objects). For 
example 

1. human (x): "x is a human" 

2. needs-to-drink(a;): "x" needs to drink 

An example of a compound statement is 




human(a;) =^ needs-to-drink(x). 
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In the above statement, =^ is the symbol used for logical implication. If we want to assert 
that every human needs to drink, we can write 

Va;(human(a;) =^ needs-to-drink(a;)); 

This is read: "for every x, if a; is a human then x needs to drink" . 
If we want to assert that some human needs to drink we write 

3a; (human (x) =^ needs-to-drink(x)); 

This is read: "for some x, if x is a human then x needs to drink" . 

We often denote statements (also called propositions or (logical) formulae) using letters, 
such SiS A,B, P, Q, etc., typically upper-case letters (but sometimes greek letters, </?, ■0, etc.). 

If P and Q are statements, then their conjunction is denoted P A Q (say: P and Q), 
their disjunction denoted P V Q (say: P or Q), their implication P ^ Q or P D Q (say: if 
P then Q). Some authors use the symbol — > and write an implication as P ^ Q. We do not 
like to use this notation because the symbol — > is already used in the notation for functions 
(/: A — > B). We will mostly use the symbol 

We also have the atomic statements ± (falsity), which corresponds to false (think of it 
as the statement which is false no matter what), and the atomic statement T (truth), which 
corresponds to true (think of it as the statement which is always true). The constant ± is 
also called falsum or absurdum. Then, it is convenient to define the negation of P as P 
and to abbreviate it as -iP (or sometimes ~ P). Thus, -iP (say: not P) is just a shorthand 
for P 

Whenever necessary to avoid ambiguities, we add matching parentheses: (PAQ), (PVQ), 
(P =^ g). For example, PVQARis ambigous; it means either {PV {Q AR)) or {{PV Q) AR). 

Another important logical operator is equivalence. If P and Q are statements, then their 
equivalence, denoted P = Q (or P <^=^ Q), is an abbreviation for (P => Q) A {Q P). We 
often say "P if and only if Q" or even "P iff Q" for P = Q. As we will see shortly, to prove 
a logical equivalence, P = Q, we have to prove both implications P =^ Q and Q =^ P. 

An implication P =^ Q should be understood as an if-then statement, that is, if P is 
true then Q is also true. So, the meaning of negation is that if -iP holds then P must be 
false. Otherwise, as -iP is really P =^-L, if P were true, then ± would have to be true, but 
this is absurd. 

Of course, there are problems with the above paragraph. What does truth have to do 
with all this? What do we mean when we say "P is true" ? What is the relationship between 
truth and provability? 

These are actually deep (and tricky!) questions whose answers are not so obvious. One 
of the major roles of logic is to clarify the notion of truth and its relationship to provability. 
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We will avoid these fundamental issues by dealing exclusively with the notion of proof. So, 
the big question is: What is a proof? 

Typically, the statements that we prove depend on some set of hypotheses, also called 
premises (or assumptions) . As we shall see shortly, this amounts to proving implications of 
the form 



However, there are certain advantages in defining the notion of proof (or deduction) of a 
proposition from a set of premises. Sets of premises are usually denoted using upper-case 
greek letters such as F or A. 

Roughly speaking, a deduction of a proposition Q from a set of premises F is a finite 
labeled tree whose root is labeled with Q (the conclusion), whose leaves are labeled with 
premises from F (possibly with multiple occurrences), and such that every interior node 
corresponds to a given set of proof rules (or inference rules). Certain simple deduction trees 
are declared as obvious proofs, also called axioms. 

There are many kinds of proofs systems: Hilbert-style systems, Natural-deduction sys- 
tems, Gcntzcn scquents systems, etc. We describe a so-called natural- deduction system 
invented by G. Gentzen in the early 1930's (and thoroughly investigated by D. Prawitz in 
the mid 1960's). The major advantage of this system is that it captures quite nicely the 
"natural" rules of reasoning that one uses when proving mathematical statements. This does 
not mean that it is easy to find proofs in such a system or that this system is indeed very 
intuitive! We begin with the inference rules for implication. 

In the definition below, the expression F, P stands for the union of F and P. So, P may 
already belong to F. A picture such as 



represents a deduction tree whose root is labeled with P and whose leaves are labeled with 
propositions from A (possibly with multiples occurrences). Some of the propositions in A 
may be tagged be variables. The list of untagged propositions in A is the list of premises of 
the deduction tree. For example, in the deduction tree below. 



(Pi A P2 A • • • A P„) =^ Q. 



A 



P 



P^Q 



P 



P^(R^ S) 



P 



Q^R 



Q 



R^S 



R 



S 



no leaf is tagged, so the premises form the set 



A^{P^{R^ S),P,Q^R,P^Q}, 



with two occurrences of P, and the conclusion is S. 
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Certain inferences rules have the effect that some of the original premises may be dis- 
carded; the traditional jargon is that some premises may be discharged (or closed). This 
this the case for the inference rule whose conclusion in an implication. When one or several 
occurrences of some proposition, P, are discharged by an inference rule, these occurrences 
(which label some leaves) are tagged with some new variable not already appearing in the 
deduction tree. If x is a new tag, the tagged occurrences of P are denoted P^ and we indicate 
the fact that premises were discharged by that inference by writing x immediately to the 
right of the inference bar. For example, 

Q 

X 

P^Q 

is a deduction tree in which the premise P is discharged by the inference rule. This deduction 
tree only has Q as a premise, since P is discharged. 

What is the meaning of the horizontal bars? Actually, nothing really! Here, we are victims 
of an old habit in logic. Observe that there is always a single proposition immediately under 
a bar but there may be several propositions immediately above a bar. The intended meaning 
of the bar is that the proposition below it is obtained as the result of applying an inference 
rule to the propositions above it. For example, in 

Q^R Q 
R 

the proposition R is the result of applying the ^-elimination rule (see Definition 11.2.11 below) 
to the two premises Q ^ R and Q. Thus, the use of the bar is just a convention used by 
logicians going back at least to the 1900's. Removing the bar everywhere would not change 
anything to our trees, except perhaps reduce their readability! Since most logic books draw 
proof trees using bars to indicate inferences, we also use bars in depicting our proof trees. 

Since propositions do not arise from the vacuum but instead are built up from a set 
of atomic propositions using logical connectives (here, =^), we assume the existence of an 
"official set of atomic propositions", PS = {Pi, P2, P3, ■ ■ ■ }. So, for example. Pi =^ P2 
and Pi =^ (P2 =^ Pi) are propositions. Typically, we will use upper-case letters such as 
P, Q, R, S, A, B, C, etc., to denote arbitrary propositions formed using atoms from PS. 

Definition 1.2.1 The axioms and inference rules for implicational logic are: 

p 

The above is a concise way of denoting a tree whose leaves are labeled with P and the 
propositions in F, each of these proposition (including P) having possibly multiple occur- 
rences but at least one, and whose root is labeled with P. A more explicit form is 
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k\ ki kn 

Pi - ■ ■ ■ . Pi - • • • ■ Pi- ■ ■ ■ ■ Pi - ■ ■ ■ ■ Pii - ■ ■ ■ ■ Pn 

Pi 

where /ci, . . . , A;„ > 0, n > 1 and ki > 1 for some i with 1 < i < n. This axiom says that we 
always have a deduction of Pj from any set of premises including Pj. 

The =^ -introduction rule: 

X 

P^Q 

This inference rule says that if there is a deduction of Q from the premises in F and from 
the premise P, then there is a deduction oi P => Q from F. Note that this inference rule has 
the additional effect of discharging some occurrences of the premise P. These occurrences 
are tagged with a new variable, x, and the tag x is also placed immediately to the right of 
the inference bar. This is a reminder that the deduction tree whose conclusion is P =^ Q no 
longer has the occurrences of P labeled with x as premises. 

The =^ -elimination rule: 

F A 
P^Q P 

Q 

This rule is also known as modus ponens. 

In the above axioms and rules, F or A may be empty and P, Q denote arbitrary propo- 
sitions built up from the atoms in PS. A deduction tree is a tree whose interior nodes 
correspond to applications of the above inference rules. A proof tree is a deduction tree 
such that all its premises are discharged. The above proof system is denoted A/"^ (here, the 
subscript m stands for minimal, referring to the fact that this a bare-bone logical system). 

In words, the =^-introduction rule says that in order to prove an implication P =^ Q 
from a set of premises F, we assume that P has already been proved, add P to the premises 
in F and then prove Q from F and P. Once this is done, the premise P is deleted. This 
rule formalizes the kind of reasoning that we all perform whenever we prove an implication 
statement. In that sense, it is a natural and familiar rule, except that we perhaps never 
stopped to think about what we are really doing. However, the business about discharging 
the premise P when we are through with our argument is a bit puzzling. Most people 
probably never carry out this "discharge step" consciously, but such a process does takes 
place implicitely. 

It might help to view the action of proving an implication P ^ Q as the construction 
of a program that converts a proof of P into a proof of Q. Then, if we supply a proof of 
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P as input to this program (the proof of P ^ Q), it will output a proof of Q. So, if we 
don't give the right kind of input to this program, for example, a "wrong proof" of P, we 
should not expect that the program retun a proof of Q. However, this does not say that the 
program is incorrect; the program was designed to do the right thing only if it is given the 
right kind of input. From this functional point of view (also called, constructive), if we take 
the simphstic view that P and Q assume the truth values true and false, we should not be 
shocked that if we give as input the value false (for P), then the truth value of the whole 
implication P ^ Q is true. The program P ^ Q is designed to produce the output value 
true (for Q) if it is given the input value true (for P). So, this program only goes wrong 
when, given the input true (for P), it returns the value false (for Q). In this erroneous 
case, P ^ Q should indeed receive the value false. However, in all other cases, the program 
works correctly, even if it is given the wrong input (false for P) . 



1. Only the leaves of a deduction tree may be discharged. Interior nodes, including the 
root, are never discharged. 

2. Once a set of leaves labeled with some premise P marked with the label x has been 
discharged, none of these leaves can be discharged again. So, each label (say x) can 
only be used once. This corresponds to the fact that some leaves of our deduction trees 
get "killed off" (discharged). 

3. A proof is deduction tree whose leaves are all discharged (T is empty). This corre- 
sponds to the philosophy that if a proposition has been proved, then the validity of 
the proof should not depend on any assumptions that are still active. We may think 
of a deduction tree as an unfinished proof tree. 

4. When constructing a proof tree, we have to be careful not to include (accidently) extra 
premises that end up not beeing discharged. If this happens, we probably made a 
mistake and the redundant premises should be deleted. On the other hand, if we have 
a proof tree, we can always add extra premises to the leaves and create a new proof 
tree from the previous one by discharging all the new premises. 

5. Beware, when we deduce that an implication P ^ Q is provable, we do not prove 
that P and Q are provable; we only prove that if P is provable then Q is provable. 

The =^-elimination rule formalizes the use of auxiliary lemmas, a mechanism that we use 
all the time in making mathematical proofs. Think of P =^ Q as a lemma that has already 
been estabhshed and belongs to some data base of (useful) lemmas. This lemma says if I can 
prove P then I can prove Q. Now, suppose that we manage to give a proof of P. It follows 
from the ^-elimination rule that Q is also provable. 

Observe that in an introduction rule, the conclusion contains the logical connective as- 
sociated with the rule, in this case, =^; this jutifies the terminology "introduction". On the 
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other hand, in an ehmination rule, the logical connective associated with the rule is gone 
(although it may still appear in Q). The other inference rules for A, V, etc., will follow this 
pattern of introduction and elimination. 

Examples of proof trees. 

(a) 

px 

~p 

X 

p^p 

So, P =^ P is provable; this is the least we should expect from our proof system! 
(b) 

(P QY 

(g ^ py Q 
'r 

X 

P^R 
y 

(Q ^ P) ^ (P ^ P) 

z 

(P^Q)^ {{Q ^ P) ^ (P ^ P)) 

In order to better appreciate the difference between a deduction tree and a proof tree, 
consider the following two examples: 

1. The tree below is a deduction tree, since two its leaves are labeled with the premises 
P =^ Q and Q ^ R, that have not been discharged yet. So, this tree represents a deduction 
of P =^ P from the set of premises r = {P =^ Q,Q =^ R} but it is not a proof tree since 
r 7^ 0. However, observe that the original premise, P, labeled x, has been discharged. 

P^Q P^ 
Q^R Q 

R 

X 

P^R 

2. The next tree was obtained from the previous one by applying the =^-introduction 
rule which triggered the discharge of the premise Q =^ R labeled y, which is no longer active. 

However, the premise P ^ Q is still active (has not been discharged, yet), so the tree below 
is a deduction tree of (Q ^ P) ^ (P =^ P) from the set of premises V — {P =^ Q}. It is 
not yet a proof tree since F 7^ 0. 



1.2. INFERENCE RULES, DEDUCTIONS, THE PROOF SYSTEMS AND MQm 19 

P^Q 

(Q ny Q 

R 

X 

P^R 
y 

{Q^R)^{P^ R) 

Finally, one more application of the =^-introduction rule will discharged the premise 
P =^ Q, at last, yielding the proof tree in (b). 

(c) In the next example, the two occurrences of A labeled x are discharged simultaneously. 

(A^{B^ C)y A"" (A =^ B)y 

B^C B 
C 

X 

A^C 
y 

[A^ B)^ {A^ C) 

z 

[A^{B^ C)) {{A ^B)^{A^ C)) 

(d) In contrast to Example (c), in the proof tree below the two occurrences of A are 
discharded separately. To this effect, they are labeled differently. 

[A^{B^ C)y {A =^ Bf A^ 

B^C B 
C 

X 

y 

{A^ B)^ {A^ C) 

z 

[A^{B=^ C)) ^ {{A ^B)^{A=^ C)) 
t 

A^ (^{A^ {B^ C)) ^ {{A ^B)^{A^ C))) 

Remark: How do we find these proof trees? Well, we could try to enumerate all possible 
proof trees systematically and see if a proof of the desired conclusion turns up. Obviously, 
this is a very inefficient procedure and moreover, how do we know that all possible proof 
trees will be generated and how do we know that such a method will terminate after a finite 
number of steps (what if the proposition proposed as a conclusion of a proof is not provable)? 
This is a very difficult problem and, in general, it can be shown that there is no procedure 
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that will give an answer in all cases and terminate in a finite number of steps for all possible 
input propositions. We will come back to this point in Section 11.71 However, for the system 
A/"^, such a procedure exists, but it is not easy to prove that it terminates in all cases and 
in fact, it can take a very long time. 

What we did, and we strongly advise our readers to try it when they attempt to construct 
proof trees, is to construct the proof tree from the bottom-up, starting from the proposition 
labeling the root, rather than top-down, i.e., starting from the leaves. During this process, 
whenever we are trying to prove a proposition P ^ Q, we use the ^-introduction rule 
backward, i.e., we add P to the set of active premises and we try to prove Q from this 
new set of premises. At some point, we get stuck with an atomic proposition, say Q. Call 
the resulting deduction Vhu, note that Q is the only active (undischarged) premises of 
and the node labeled Q immediately below it plays a special role; we will call it the special 
node of Vbu- The trick is to now switch strategy and start building a proof tree top-down, 
starting from the leaves, using the ^-elimination rule. If everything works out well, we get a 
deduction with root Q, say Vtd, and then we glue this deduction Vtd to the deduction V^u in 
such a way that the root of is identified with the special node of labeled Q. We also 
have to make sure that all the discharged premises are linked to the correct instance of the 
^-introduction rule that caused them to be discharged. One of the difficulties is that during 
the bottom-up process, we don't know how many copies of a premise need to be discharged 
in a single step. We only find out how many copies of a premise need to be discharged during 
the top-down process. 

Here is an illustration of this method for our third example. At the end of the bottom-up 
process, we get the deduction tree Vbu- 

{A^{B^ c)y {A B)y C 

C 

X 

A^C 
y 

[A^ B)^ {A^ C) 

2 

[A^{B^ C)) {{A ^ B)^{A^ C)) 

At the end of the top-down process, we get the deduction tree Vtd- 

A^(B^C) A A^B A 

B^C B 

C 

Finally, after glueing Vtd on top of V^u (which has the correct number of premises to be 
discharged), we get our proof tree: 
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[A^{B^ C)y A" {A =^ B)y 



B^C B_ 

C 

X 



A^C 



{A^B)^{A^ C) 



[A^{B^ C)) {{A ^B)^{A^ C)) 

Let us return to the functional interpretation of implication by giving an example. The 
proposition P =^ {{P ^ Q) ^ Q) has the following proof: 

(p =^ Q)^ py 

Q 



{P^Q)^Q 
P^iiP^Q)^ Q) 
Now, say P is the proposition R, which has the proof 

K 
R 



R^ R 



Using =^-ehmination, we obtain a proof oi {{R ^ R) =^ Q) => Q from the proof of 
{R^ R)^ {{{R ^ R)^Q)^Q) and the proof of R ^ R: 

{{R ^R)^ QY {R ^ R)y 

Q 



X R_ 

((p =^ i?) =^ g) ^ g ^ 



{R^ R)^{{{R^ R)^Q)=^Q) R^R 
{{R ^R)=^Q)^Q 

Note that the above proof is redundant. A more direct proof can be obtained as follows: 
Undo the last =^-introduction in the proof oi {R ^ R) ^ {{{R ^ R) ^ Q) ^ Q): 

((R ^R)^QY R^R 

Q 
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and then glue the proof of i? ^ i? on top of the leaf R ^ R, obtaining the desired proof of 
{{R ^B)^Q)^Q: 

R 

2 

((i? ^ R)^ Q)'^ R^R 
Q 

X 

((R ^R)^Q)^Q 

In general, one has to exercise care with the label variables. It may be necessary to re- 
name some of these variables to avoid clashes. What we have above is an example of proof 
substitution also called proof normalization. We will come back to this topic in Section [TTTl 

The process of discharging premises when constructing a deduction is admittedly a bit 
confusing. Part of the problem is that a deduction tree really represents the last of a sequence 
of stages (corresponding to the application of inference rules) during which the current set 
of "active" premises, that is, those premises that have not yet been discharged (closed, 
cancelled) evolves (in fact, shrinks). Some mechanism is needed to keep track of which 
premises are no longer active and this is what this business of labeling premises with variables 
achieves. Historically, this is the first mechanism that was invented. However, Gentzen (in 
the 1930's) came up with an alternative solution which is mathematically easier to handle. 
Moreover, it turns out that this notation is also better suited to computer implementations, 
if one wishes to implement an automated theorem prover. 

The point is to keep a record of all undischarged assumptions at every stage of the 
deduction. Thus, a deduction is now a tree whose nodes are labeled with expressions of the 
form r — i> P, called sequents, where P is a proposition, and F is a record of all undischarged 
assumptions at the stage of the deduction associated with this node. 

During the construction of a deduction tree, it is necessary to discharge packets of as- 
sumptions consisting of one or more occurrences of the same proposition. To this effect, it is 
convenient to tag packets of assumptions with labels, in order to discharge the propositions 
in these packets in a single step. We use variables for the labels, and a packet labeled with x 
consisting of occurrences of the proposition P is written as x: P. Thus, in a sequent T —>■ P, 
the expression T is any finite set of the form xi : Pi, . . . ,Xm- Pm, where the Xi are pairwise 
distinct (but the Pi need not be distinct). Given F = xi : Pi, . . . ,Xm- Pm, the notation 
r,x: P is only well defined when x ^ Xi for all i, 1 < i < m, in which case it denotes the 
set Xi . Pi, . . . , Xffi . Pjn, X . p. 

Using sequents, the axioms and rules of Definition 11.2.21 are now expressed as follows: 

Definition 1.2.2 The axioms and inference rules of the system AfQ^ {implicational logic, 
Gentzen- sequent style (the Q in MQ stands for Gentzen)) are listed below: 

V,x: P ^ P 
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r,x: P 



r^p^Q r^p 



-intra) 
{^-elim) 



In an application of the rule {^-intra), observe that in the lower sequent, the proposition 
P (labeled x) is deleted from the hst of premises occurring on the left-hand side of the arrow- 
in the upper sequent. We say that the proposition P which appears as a hypothesis of 
the deduction is discharged (or closed). It is important to note that the ability to label 
packets consisting of occurrences of the same proposition with different labels is essential, in 
order to be able to have control over which groups of packets of assumptions are discharged 
simultaneously. Equivalently, we could avoid tagging packets of assumptions with variables 
if we assumed that in a sequent F — > C, the expression F, also called a cantext, is a multiset 
of propositions. 

Below we show a proof of the third example given above in our new system. Let 

T = x: A^ {B ^C),y: A^ B,z: A. 



r^A^jB^c) r^A r^A^B t^a 

B^C F^^ 

x: A^ {B ^ C),y: A^ B,z: A^ C 

x: A^ {B ^ C),y: A^ B ^ A^ C 

x: A ^ {B ^ C) ^ {A ^ B) ^ {A ^ C) 

^(A^{B^ C)) {{A ^B)^{A^ C)) 



In principle, it does not matter which of the two systems J\fj^ or NQ^ we use to construct 
deductions; it is a matter of taste. My experience is that I make fewer mistakes with the 
Gentzen-sequent style system MQ^. 

We now describe the inference rules deahng with the connectives A, V and ±. 



1.3 Adding A, V, _L; The Proof Systems A/'^^'^'^'^ and 

Recall that -iP is an abbreviation for P =^_L. 
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Definition 1.3.1 The axioms and inference rules for (propositional) classical logic are: 
Axioms: 



The ^-introduction rule: 



The =^ -elimination rule: 



The ^-introduction rule: 



The /\- elimination rule: 



The M -introduction rule: 



The y -elimination rule: 



The elimination rule: 



Q 



P^Q 



A 



P^Q P 
Q 

r A 

P_ Q_ 

PAQ 



PAQ PAQ 
P Q 

r £ 
p Q 



PVQ PVQ 

r A, A, Qy 



pyQ R R 
R 

r 
± 
p 
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The pr 00 f-hy- contradiction rule (also known as reductio ad absurdum rule, for short 
RAA): 

L 

X 



Since -iP is an abbreviation for P =^±, the -i-introduction rule is a special case of the 
-introduction rule (with Q —A-). However, it is worth stating it explicitly: 

The -I -introduction rule: 

Ypx 

X 

^p 



Similarly, the -i-elimination rule is a special case of =^-elimination applied to 
nP (= P and P: 

The -I -elimination rule: 

r A 
^p p 

± 



In the above axioms and rules, F, A or A may be empty, P,Q,R denote arbitrary propo- 
sitions built up from the atoms in PS and all the premises labeled x are discharged. A 
deduction tree is a tree whose interior nodes correspond to applications of the above infer- 
ence rules. A proof tree is a deduction tree such that all its premises are discharged. The 
above proof system is denoted A/c^'^'^'"*" (here, the subscript c stands for classical). 

The system obtained by removing the proof-by-contradiction (RAA) rule is called (prepo- 
sitional) intuitionistic logic and is denoted A^"*'^'^'"*". The system obtained by deleting both 
the ±-elimination rule and the proof-by-contradiction rule is called (prepositional) minimal 
logic and is denoted A/"^'^'^'"*". 



The version of A/c^'^'^'"*" in terms of Gentzen sequents is the following: 

Definition 1.3.2 The axioms and inference rules of the system A/'^f"'^'^'^ (of prepositional 
classical logic, Gentzen-sequent style) are listed below: 

r,x: P ^ P 
r,x:P-^Q .... 
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^ {=>-elim) 



r^PAQ 



{A-intro) 



^ ^ (A-elim) — --^ {A-elim) 



r^p ' ' r^Q 
r ^ p r ^ Q 

'y-intro) — — -- iy-intro) 



r^pvQ ' ' r^pvg 

r^Pvg r,x:P^R V.y.Q^R 



V^R 

r ^± 
r ^ p 

T,x: ^P ^± 

r ^ p 

r,x:P^± 

r^^p ^ 
r^^p r ^ p 



{V-elim) 



{J--elim) 

{by- contra) 
-i-introduction) 

(-i-elimination) 



Since the rule {-L-elim) is trivial (does nothing) when P =±, from now on, we will assume 

that P 7^±. Propositional minimal logic, denoted A/"^^'^'^'^, is obtained by dropping the 
{±-elim) and [hy-contra) rules. Propositional intuitionistic logic, denoted NQ"^'^'^'^ , is 
obtained by dropping the {hy-contra) rule. 



When we say that a proposition, P, is provable from F, we mean that we can construct 
a proof tree whose conclusion is P and whose set of premises is F, in one of the systems 
jVp"*'^'^'-*" or A/'^^''^'^'"'". Therefore, when we use the word "provable" unqualified, wc mean 
provable in classical logic. If P is provable from F in one of the intuitionistic systems A/^"*'^'^'"*" 
or AfQf''^'^''^ , then we say intuitionistically provable (and similarly, if P is provable from F 
in one of the systems A/"^'^'"^'"*" or A/"^^'^'^'"*", then we say provable in minimal logic). When 
P is provable from F, most people write F h P, or h F — > P, sometimes with the name of 
the corresponding proof system tagged as a subscript on the sign h if necessary to avoid 
ambiguities. When F is empty, we just say P is provable (provable in intuitionistic logic, 
etc.) and write h P. 

We treat logical equivalence as a derived connective, that is, we view P = Q as an 
abbreviation for {P ^ Q) A{Q ^ P). In view of the inference rules for A, we see that to 
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prove a logical equivalence P = Q, we just have to prove both implications P =^ Q and 
Q^P. 

In view of the _L-elimination rule, the best way to interpret the provability of a negation, 
-iP, is as "P is not provable". Indeed, if -iP and P were both provable, then ± would be 
provable. So, P should not be provable if -iP is. This is not the usual interpretation of 
negation in terms of truth values, but it turns out to be the most fruitful. Beware that if P 
is not provable, then -iP is not provable in general! There are plenty of propositions such 
that neither P nor -iP is provable (for instance, P, with P an atomic proposition). 

Let us now make some (much-needed) comments about the above inference rules. There 
is no need to repeat our comments regarding the ^-rules. 

The A-introduction rule says that in order to prove a conjunction P A Q from some 
premises F, all we have to do is to prove both that P is provable from F and that Q is 
provable from F. The A-elimination rule says that once wc have proved P A Q from F, then 
P (and Q) is also provable from F. This makes sense intuitively as PA Q is "stronger" than 
P and Q separately (P A Q is true iff both P and Q are true). 

The V-introduction rule says that if P (or Q) has been proved from F, then P V Q is 
also provable from F. Again, this makes sense intuitively as P V Q is "weaker" than P and 
Q. The V-elimination rule formalizes the proof-by- cases method. It is a more subtle rule. 
The idea is that if we know that in the case where P is already assumed to be provable and 
similarly in the case where Q is already assumed to be provable that we can prove R (also 
using premises in F), then if P VQ is also provable from F, as we have "covered both cases", 
it should be possible to prove R from F only (i.e., the premises P and Q are discarded). 

The ±-elimination rule formalizes the principle that once a false statement has been 
established, then anything should be provable. 

The proof-by-contradiction rule formalizes the method of proof by contradiction! That 
is, in order to prove that P can be deduced from some premises F, one may assume the 
negation, -iP, of P (intuitively, assume that P is false) and then derive a contradiction from 
F and -iP (i.e., derive falsity). Then, P actually follows from F without using ->P as a 
premise, i.e., ->P is discharged. 

Most people, I believe, will be comfortable with the rules of minimal logic and will agree 
that they constitute a "reasonable" formalization of the rules of reasoning involving A 
and V. Indeed, these rules seem to express the intuitive meaning of the connectives A 
and V. However, some may question the two rules _L-elimination and proof- by-contradiction. 
Indeed, their meaning is not as clear and, certainly, the proof-by-contradiction rule introduces 
a form of indirect reasoning that is somewhat worrisome. 

The problem has to do with the meaning of disjunction and negation and more gener- 
ally, with the notion of constructivity in mathematics. In fact, in the early 1900's, some 
mathematicians, especially L. Brouwer (1881-1966), questioned the validity of the proof-by- 
contradiction rule, among other principles. Two specific cases illustrate the problem, namely. 
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the propositions 

Py^P and ^^P =^ P. 

As we will see shortly, the above propositions are both provable in classical logic. Now, 
Brouwcr and some mathematicians belonging to his school of thoughts (the so-called "intu- 
itionsists" or "constructivists" ) advocate that in order to prove a disjunction, P \/ Q (from 
some premises F) one has to either exhibit a proof of P or a proof or Q (from F). However, it 
can be shown that this fails for P V -iP. The fact that P V -iP is provable (in classical logic) 
does not imply that either P is provable or that -iP is provable! That P V -iP is provable is 
sometimes called the principle of the excluded middlel In intuitionistic logic, P V -iP is not 
provable. Of course, if one gives up the proof-by-contradiction rule, then fewer propositions 
become provable. On the other hand, one may claim that the propositions that remain 
provable have more constructive proofs and thus, feels on safer grounds. 

A similar controversy arises with -i-iP =^ P. If we give up the proof-by-contradiction 
rule, then this formula is no longer provable, i.e., -i-iP is no longer equivalent to P. Perhaps 
this relates to the fact that if one says 

" I don't have no money" 

then this does not mean that this person has money! (Similarly with "I don't get no satis- 
faction", ... ). However, note that one can still prove P =^ -i-iP in minimal logic (try doing 
it!). Even stranger, ^^^P => ^P is provable in intuitionistic (and minimal) logic, so -i-i-iP 
and -iP are equivalent intuitionistically! 

Remark: Suppose we have a deduction 

r,^p 
± 

as in the proof by contradiction rule. Then, by -i-introduction, we get a deduction of -i-iP 
from F: 

F,^P^ 
± 

X 

^^p 

So, if we knew that -i-iP was equivalent to P (actually, if we knew that -i-iP ^ P is 
provable) then the proof by contradiction rule would be justified as a valid rule (it follows 
from modus ponens) . We can view the proof by contradiction rule as a sort of act of faith 
that consists in saying that if we can derive an inconsistency (i.e., chaos) by assuming the 
falsity of a statement P, then P has to hold in the first place. It not so clear that such an 
act of faith is justified and the intuitionists refuse to take it! 

Constructivity in mathematics is a fascinating subject but it is a topic that is really 
outside the scope of this course. What we hope is that our brief and very incomplete 
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discussion of constructivity issues made the reader aware that the rules of logic are not cast 
in stone and that, in particular, there isn't only one logic. 

We feel safe in saying that most mathematicians work with classical logic and only few 

of them have reservations about using the proof-by-contradiction rule. Nevertherless, in- 
tuitionistic logic has its advantages, especially when it comes to proving the correctess of 
programs (a branch of computer science!). We will come back to this point several times in 
this course. 

In the rest of this section, we make further useful remarks about (classical) logic and give 
some exphcit examples of proofs illustrating the inference rules of classical logic. We begin 
by proving that P V -iP is provable in classical logic. 

Proposition 1.3.3 The proposition P V -iP is provable in classical logic. 

Proof. We prove that P y {P =^-L) is provable by using the proof-by-contradiction rule as 
shown below: 

px 

{{py {P ^l_))^l_)y py(P^±) 
I 

X 

p ^± 

{{py {P ^±)) ^±)y Pv(P^±) 

1*1 y (by-contra) 

P V (P 

□ 

Next, we consider the equivalence of P and -i-iP. 

Proposition 1.3.4 The proposition P =>■ -i-iP is provable in minimal logic. The proposition 
-i-iP ^ P is provable in classical logic. Therefore, in classical logic, P is equivalent to -i-iP. 

Proof. We leave that P =^ -i-iP is provable in minimal logic as an exercise. Below is a proof 
of -i-iP =^ P using the proof-by-contradiction rule: 

((p^±)^±)y {p^±y 

Jr X (by-contra) 
P 

y 

{{P ^ p 

□ 

The next proposition shows why _L can be viewed as the "ultimate" contradiction. 
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Proposition 1.3.5 In intuitionistic logic, the propositions 1. and P A-iP are equivalent for 
all P. Thus, J- and P A -iP are also equivalent in classical propositional logic 

Proof. We need to show that both _L=^ (P A -iP) and (P A -iP) =^_L are provable in 
intuitionistic logic. The provability of ±^ (P A -iP) is an immediate consequence or ±- 
ehmination, with F = 0. For (P A -iP) =^-L, we have the following proof: 

(p A ^py (p A -^py 
^p p 
± 

X 

(P A -P) ^± □ 

So, in intuitionistic logic (and also in classical logic), _L is equivalent to P A -iP for all 
P. This means that _L is the "ultimate" contradiction, it corresponds to total inconsistency. 
By the way, we could have the bad luck that the system (or A/^"*'^'^'"*" or even 

jV^'^'^'"*") is inconsistent, that is, that ± is provable! Fortunately, this is not the case, 
although this hard to prove. (It is also the case that P V -iP and -i-iP =^ P are not 
provable in intuitionistic logic, but this too is hard to prove!) 



1.4 Clearing Up Differences Between 

-i-introduction, J_-elimination and RAA 

The differences between the rules, -i-introduction, ±-elimination and the proof by contradic- 
tion rule (RAA) are often unclear to the uninitiated reader and this tends to cause confusion. 
In this section, wc will try to clear up some common misconceptions about these rules. 

Confusion 1. Why is RAA not a special case of -i- introduction? 

F, P^ F, ^P^ 



^ x(-i-intro) ^ a:(RAA) 

^P P 

The only apparent difference between -i-introduction (on the left) and RAA (on the right) is 
that in RAA, the premise P is negated but the conclusion is not, whereas in -i-introduction 
the premise P is not negated but the conslusion is. 

The important difference is that the conclusion of RAA is not negated. If we had apphed 
-i-introduction instead of RAA on the right, we would have obtained 



F,-P 



X 



~'~ X (-1-intro) 



nP 
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where the conclusion would have been -i-iP as opposed to P. However, as we already said 
earher, -i-iP =^ P is not provable intuitionistically. Consequenly, RAA is not a special case 
of -i-introduction. 

Confusion 2. Is there any difference between ±-elimination and RAA? 

r r, ^P^ 

^ (±-elim) ^ x(RAA) 

P P 



The difference is that ±-elimination does not discharge any of its premises. In fact, RAA 
is a stronger rule which implies ±-elimination as we now demonstate. 

RAA implies _L-elimination. 

Suppose we have a deduction 

r 

± 



Then, for any proposition P, we can add the premise -iP to every leaf of the above deduction 
tree and we get the deduction tree 

r,^p 



We can now apply RAA to get the following deduction tree of P from F (since -iP is 
discharged), and this is just the result of ±-elimination: 

r,^p^ 

^ X (RAA) 



The above considerations also show that RAA is obtained from ±-elimination by adding 
the new rule of -i-i- elimination: 



-elimination) 



Some authors prefer adding the -i-i-elimination rule to intuitionistic logic instead of RAA 
in order to obtain classical logic. As we just demonstrated, the two additions are equivalent: 
by adding either RAA or -i-i-elimination to intuitionistic logic, we get classical logic. 
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There is another way to obtain RAA from the rules of intuitionistic logic, this time, using 
the propositions of the form P V -iP. We saw in Proposition 11.3.31 that all formulae of the 
form P V -iP are provable in classical logic (using RAA). 

Confusion 3. Are propositions of the form P V -iP provable in intuitionistic logic? 

The answer is no, which may be disturbing to some readers. In fact, it is quite difficult 
to prove that propositions of the form P V -iP are not provable in intuitionistic logic. One 
method consists in using the fact that intuitionistic proofs can be normalized (see Section [m 
for more on normalization of proofs). Another method uses Kripke models (see van Dalen 

m)- 

Part of the difficulty in understanding at some intuitive level why propositions of the 
form P V -iP are not provable in intuitionistic logic is that the notion of truth based on the 
truth values true and false is deeply rooted in all of us. In this frame of mind, it seems 
ridiculous to question the provability of P V -iP, since its truth value is true whether P is 
assigned the value true or false. Classical two-valued truth value semantics is too crude for 
intuitionistic logic. 

Another difficulty is that it is tempting to equate the notion of truth and the notion of 
provability. Unfortunately, because classical truth value semantics is too crude for intuition- 
istic logic, there are propositions that are universally true (i.e., they evaluate to true for 
all possible truth assignments of the atomic letters in them) and yet they are not provable 
intuitionistically. The propositions P V -iP and -i-iP P are such examples. 

One of the major motivations for advocating intuitionistic logic is that it yields proofs 
that are more constructive than classical proofs. For example, in classical logic, when we 
prove a disjunction P V Q, we generally can't conclude that either P or Q is provable, as 
examplified by P V -iP. A more interesting example involving a non-constructive proof of a 
disjunction will be given in Section [T31 But, in intuitionistic logic, from a proof of P V Q, 
it is possible to extract either a proof of P or a proof or Q (and similarly for existential 
statements, see Section [TT^ . This property is not easy to prove. It is a consequence of the 
normal form for intuitionistic proofs (see Section [LT]) . 

In brief, besides being a fun intellectual game, intuitionistic logic is only an interesting 
alternative to classical logic if we care about the constructive nature of our proofs. But 
then, we are forced to abandon the classical two-valued truth value semantics and adopt 
other semantics such as Kripke semantics. If we do not care about the constructive nature 
of our proofs and if we want to stick to two- valued truth value semantics, then we should 
stick to classical logic. Most people do that, so don't feel bad if you are not comfortable 
with intuitionistic logic! 

One way to gauge how intuitionisic logic differs from classical logic is to ask what kind 
of propositions need to be added to intuitionisic logic in order to get classical logic. It turns 
out that if all the propositions of the form P V -iP are considered to be axioms, then RAA 
follows from some of the rules of intuitionistic logic. 
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RAA holds in Intuitionistic logic + all cixioms P V -iP. 

The proof involves a subtle use of the _L-elimination and V-elimination rules which may 
be a bit puzzling. Assume, as we do when when use the proof by contradiction rule (RAA) 
that we have a deduction 

± 



Here is the deduction tree demonstrating that RAA is a derived rule: 



px 

py^P ~P 
P 

At first glance, the rightmost subtree 

v,^py 

Jr (_L-elim) 
P 



v,^py 

^ (±-elim) 
p 

x,y (V-elim) 



appears to use RAA and our argument looks circular! But this is not so because the premise 
-iP labeled y is not discharged in the step that yields P as conclusion; the step that yields P 
is a _L-elimination step. The premise -iP labeled y is actually discharged by the V-elimination 
rule (and so is the premise P labeled x). So, our argument establishing RAA is not circular 
after all! 

In conclusion, intuitionistic logic is obtained from classical logic by taking away the proof 
by contradiction rule (RAA). In this more restrictive proof system, we obtain more construc- 
tive proofs. In that sense, the situation is better than in classical logic. The major drawback 
is that we can't think in terms of classical truth value semantics anymore. 

Conversely, classical logic is obtained from intuitionistic logic in at least three ways: 

1. Add the proof by contradiction rule (RAA). 

2. Add the -i-i-elimination rule. 

3. Add all propositions of the form P V -iP as axioms. 
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1.5 Other Rules of Classical Logic and Examples of 
Proofs 

In classical logic, we have the de Morgan laws: 

Proposition 1.5.1 The following equivalences (de Morgan laws) are provable in classical 
logic: 

^(PAQ) = V^Q 
^{PyQ) = -PA-Q. 

In fact, ->{P\/Q) = -iPA-iQ and (-iPV-iQ) ^ -'{PAQ) are provable in intuitionistic logic. 
The proposition (P A -iQ) =^ -i(P =^ Q) is provable in intuitionistic logic and -i(P =^ Q) ^ 

(P A -iQ) is provable in classical logic. Therefore, -i(P =^ Q) and P A -iQ are equivalent 
in classical logic. Furthermore, P Q and -iP V Q are equivalent in classical logic and 
(-iP y Q) ^ {P ^ Q) is provable in intuitionistic logic. 

Proof. Here is an intuitionistic proof of (-iP \/ Q) =^ {P =^ Q): 

^pz px 

I py Q* 



Q Q 

X y 



^pyQY P^Q P^Q 

z,t 

P^Q 



(-P V g) ^ (p ^ g) 

Here is a classical proof of {P ^ Q) =^ (-iP V Q): 



{-n{-nPyQ))y ^pyQ 
{P => QY 

g 



^ X RAA 



H-pvg))f -pvg 

^ y RAA 



nPvg 



(p =^ g) ^ (-P V g) 
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The other proofs are left as exercises. □ 

Propositions 1 1 . 3 .11 and [T3TT] show a property that is very specific to classical logic, namely, 
that the logical connectives A, V, -i are not independent. For example, we have 
P A Q = ~i(~i-P V -iQ), which shows that A can be expressed in terms of V and -i. In 
intuitionistic logic, A and V cannot be expressed in terms of each other via negation. 

The fact that the logical connectives A, V, -i are not independent in classical logic 
suggests the following question: Are there propositions, written in terms of =^ only, that are 
provable classically but not provable intuitionistically? 

The answer is yes! For instance, the proposition ((P ^ Q) ^ P) ^ P (known as Pierce's 
law) is provable classically (do it) but it can be shown that it is not provable intuitionistically. 

In addition to the proof by cases method and the proof by contradiction method, we also 
have the proof by contrapositive method valid in classical logic: 

Proof by contrapositive rule: 

^p 

X 

P^Q 

This rule says that in order to prove an implication P ^ Q (from F), one may assume 
-iQ as proved, and then deduce that -iP is provable from F and -iQ. This inference rule is 
valid in classical logic because we can construct the following proof: 

-nP py 

^ X (by-contra) 

Q 

y 

P^Q 

We will now give some explicit examples of proofs illustrating the proof principles that 
we just discussed. 

Recall that the set of integers is the set 

Z = {... ,-2,-l,0,l,2,---} 
and that the set of natural numbers is the set 

N={0,1,2,---}. 

(Some authors exclude from N. We don't like this discrimination against zero.) An integer 
is even if it is divisible by 2, that is, if it can be written as 2k, where k E 'L. An integer 
is odd if it is not divisible by 2, that is, if it can be written as 2 A; + 1, where A; G Z. The 
following facts are essentially obvious: 
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(a) The sum of even integers is even. 

(b) The sum of an even integer and of an odd integer is odd. 

(c) The sum of two odd integers is even. 

(d) The product of odd integers is odd. 

(e) The product of an even integer with any integer is even. 

Now, we prove the following fact using the proof by cases method. 

Proposition 1.5.2 Let a,b,c be odd integers. For any integers p and q, if p and q are not 
both even, then 

ap^ + bpq + cq^ 

is odd. 

Proof . We consider the three cases: 

1. p and q are odd. In this case as a, b and c are odd, by (d) all the products ap^, bpq and 
cq^ are odd. By (c), ap^ + bpq is even and by (b), ap^ + bpq + cq^ is odd. 

2. p is even and q is odd. In this case, by (e), both ap^ and bpq are even and by (d), cq^ 
is odd. But then, by (a), ap^ + bpq is even and by (b), ap^ + bpq + cq^ is odd. 

3. p is odd and q is even. This case is analogous to the previous case, except that p and 
q are interchanged. The reader should have no trouble filling in the details. 

Since all three cases exhaust all possibilities for p and q not to be both even, the proof is 
complete by the V-elimination rule (applied twice). □ 

The set of rational numbers Q consists of all fractions p/q, where p,q ^ with q 0. 
We now use Proposition 11.5.21 and the proof by contradiction method to prove 

Proposition 1.5.3 Let a,b,c be odd integers. Then, the equation 

aX^ + bX + c = 

has no rational solution X . 

Proof. We proceed by contradiction (by this, we mean that we use the proof by contradiction 
rule). So, assume that there is a rational solution X = p/q. We may assume that p and 
q have no common divisor, which implies that p and q are not both even. As g 7^ 0, if 
aX^ + bX + c = 0, then by multiplying by g^, we get 



ap'^ + bpq + cq^ = 0. 
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However, as p and q are not both even and a, b, c are odd, we know from Proposition 11.5.21 
that ap^ + hpq + cq^ is odd, that is, at least 1. This contradicts the fact that + hpq + cq^ = 
and thus, finishes the proof. □ 

As as example of the proof by contrapositive method, we prove that if an integer is 
even, then n must be even. 

Observe that if an integer is not even then it is odd (and vice- versa). Thus, the contra- 
positive of our statement is: If n is odd, then is odd. But, to say that n is odd is to say 
that n = 2fc + 1 and then, = {2k + if = 4k^ + ik + 1 = 2{2k^ + 2k) + 1, which shows 
that is odd. 

A real number a G M is said to be irrational if it cannot be expressed as a number in Q 
(a fraction). The reader should prove that a/2 is irrational by adapting the arguments used 
in the two previous propositions. 

Remark: Let us return briefly to the issue of constructivity in classical logic, in particular 
when it comes to disjunctions. Consider the question: are there two irrational real numbers 
a and h such that is rational? Here is a way to prove that this indeed the case. Consider 

the number a/2 . If this number is rational, then a = \/2 and h = \/2 is an answer to our 
question (since we already know that \/2 is irrational). Now, observe that 

(72^^)^ = V2^'''^ = ^/2 =2 is rational! 

Thus, if \/2 is irrational, then a = \/2 and h = \/2 is an answer to our question. So, we 
proved that 

{\/2 is irrational and \/2 is rational) or 

{Vr and V2 are irrational and (v^ )^ is rational). 

However, the above proof does not tell us whether \/2 is rational or not! 

We see one of the shortcomings of classical reasoning: certain statements (in particular, 
disjunctive or existential) are provable but their proof does provide an explicit answer. It is 
in that sense that classical logic is not constructive. 

Many more examples of non-constructive arguments in classical logic can be given. 

We now add quantifiers to our language and give the corresponding inference rules. 

1.6 Adding Quantifiers; The Proof Systems 7V^=^^^'^^'^, 

^^=^,A,V,V,3,± 

As we mentioned in Section II. atomic propositions may contain variables. The intention 
is that such variables correspond to arbitrary objects. An example is 



human(x) ^ needs-to-drink(x). 
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Now, in mathematics, we usually prove universal statements, that is statement that hold 
for all possible "objects", or existential statement, that is, statement asserting the existence 
of some object satisfying a given property. As we saw earlier, we assert that every human 
needs to drink by writing the proposition 

Vx(human(x) ^ needs-to-drink(x)). 

Observe that once the quantifier V (pronounced "for all" or "for every") is apphed to the 
variable x, the variable x becomes a place-holder and replacing a; by y or any other variable 
does not change anything. What matters is the locations to which the outer x points to in 
the inner proposition. We say that x is a bound variable (sometimes a "dummy variable"). 

If we want to assert that some human needs to drink we write 

3x(human(x) =^ needs-to-drink(x)); 

Again, once the quantifier 3 (pronounced "there exists") is applied to the variable x, the 
variable x becomes a place-holder. However, the intended meaning of the second proposition 
is very different and weaker than the first. It only asserts the existence of some object 
satisfying the statement 

human(x) =^ needs-to-drink(x). 
Statements may contain variables that are not bound by quantifiers. For example, in 

V?/ parent (x, y) 

the variable y is bound but the variable x is not. Here, the intended meaning of parent (x, y) 
is that a; is a parent of y. Variables that are not bound are called free. The proposition 

\/y3x parent(a;, y), 

which contains only bound variables in meant to assert that every y has some parent x. Typi- 
cally, in mathematics, we only prove statements without free variables. However, statements 
with free variables may occur during intermediate stages of a proof. 

The intuitive meaning of the statement \fxP is that P holds for all possible objects x and 
the intuitive meaning of the statement 3xP is that P holds for some object x. Thus, we see 
that it would be useful to use symbols to denote various objects. For example, if we want 
to assert some facts about the "parent" predicate, we may want to introduce some constant 
symbols (for short, constants) such as "Jean" , "Mia" , etc. and write 

parent (Jean, Mia) 

to assert that Jean is a parent of Mia. Often, we also have to use function symbols (or 
operators, constructors) , for instance, to write statement about numbers: +, *, etc. Using 
constant symbols, function symbols and variables, we can form terms, such as 

{x^ + l){3*y + 2). 
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In addition to function symbols, we also use predicate symbols, which are names for atomic 
properties. We have already seen several examples of predicate symbols: "human" , "parent" . 
So, in general, when we try to prove properties of certain classes of objects (people, numbers, 
strings, graphs, etc.), we assume that we have a certain alphabet consisting of constant 
symbols, function symbols and predicate symbols. Using these symbols and an infinite 
supply of variables (assumed distinct from the variables which we use to label premises) we 
can form terms and predicate terms. We say that we have a (logical) language. Using this 
language, we can write compound statements. 

Let us be a little more precise. In a first-order language, L, in addition to the logical 
connectives, =^, A, V, -i, ±, V and 3, we have a set, L, of nonlogical symbols consisting of 

(i) A set CS of constant symbols, ci, C2, . . . ,. 

(ii) A set FS of function symbols, /i, /2, • • • Each function symbol, /, has a rank, Uf > 1, 
which is the number of arguments of /. 

(iii) A set PS of predicate symbols, Pi,P2, ■ ■ ■ ,■ Each predicate symbol, P, has a rank. 
Up > 0, which is the number of arguments of P. Predicate symbols of rank are 
propositional letters, as in earlier sections. 

(iv) The equality predicate, =, is added to our language when we want to deal with equa- 
tions. 

(v) First-order variables, ti,t2, . . . , used to form quantified formulae. 

The difference between function symbols and predicate symbols is that function symbols 
are interpreted as functions defined on a structure (for example, addition, +, on N), whereas 
predicate symbols are interpreted as properties of objects, that is, they take the value true 
or false. An example is the language of Peano arithmetic, L = {0,5',-!-,*,=}. Here, the 
intended structure is N, is of course zero, S is interpreted as the function S{n) — n + 1, 
the symbol -|- is addition, * is multiplication and = is equality. 

Using a first-order language, L, we can form terms, predicate terms and formulae. The 
terms over L are the following expressions: 

(i) Every variable, t, is a term; 

(ii) Every constant symbol, c G CS, is a term; 

(iii) If / G FS is a function symbol taking n arguments and ti, . . . ,t„ are terms already 
constructed, then /(ti, . . . , r„) is a term. 

The predicate terms over L are the following expressions: 

(i) If P G PS is a predicate symbol taking n arguments and ri, . . . , r„ are terms already 
constructed, then P{ti, . . . , r„) is a predicate term. When n — 0, the predicate symbol, 
P, is a predicate term called a propositional letter. 
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(ii) When we allow the equality predicate, for any two terms ti and T2, the expression 
Ti = r2 is a predicate term. It is usually called an equation. 

The (first- order) formulae over L are the following expressions: 

(i) Every predicate term, P{ti, . . . ,Tn), is an atomic formula. This includes all proposi- 
tional letters. We also view ± (and sometimes T) as an atomic formula. 

(ii) When we allow the equality predicate, every equation, ti = T2, is an atomic formula. 

(iii) If P and Q are formulae already constructed, then P ^ Q, P A Q, P y Q, -iP are 
compound formulae. We treat P = Q as an abbreviation for {P ^ Q) A {Q ^ P), as 
before. 

(iv) If P is a formula already constructed and t is any variable, then VtP and 3tP are 
compound formulae. 

All this can be made very precise but this is quite tedious. Our primary goal is to explain 
the basic rules of logic and not to teach a full-fledged logic course. We hope that our intuitive 
explanations will suffice and we now come to the heart of the matter, the inference rules for 
the quantifiers. Once again, for a complete treatment, readers are referred to Gallier [T8] 
van Dalen [12] or Huth and Ryan |30j . 

Unlike the rules for V, A and _L, which are rather straightforward, the rules for quan- 
tifiers are more subtle due the presence of variables (occurring in terms and predicates). We 
have to be careful to forbid inferences that would yield "wrong" results and for this we have 
to be very precise about the way we use free variables. More specifically, we have to exercise 
care when we make substitutions of terms for variables in propositions. For example, say we 
have the predicate "odd" , intended to express that a number is odd. Now, we can substitute 
the term {2y + 1)^ for x in odd(x) and obtain 

odd((2|/+l)2). 

More generally, if P(ti, t2, . . . , tn) is a statement containing the free variables ti, . . . , t„ and 
if Ti, . . . , r„ are terms, we can form the new statement 

P[ri/ti, . . .,Tn/tn] 

obtained by substituting the term Ti for all free occurrences of the variable ti, for i = 1, . . . ,n. 
By the way, we denote terms by the greek letter r because we use the letter t for a variable 
and using t for both variables and terms would be confusing; sorry! 

However, if P(ti,t2, . . . ,tn) contains quantifiers, some bad things can happen, namely, 
some of the variables occurring in some term Xj may become quantified when Tj is substituted 
for tj. For example, consider 

\fx3y P{x,y,z) 
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which contains the free variable z and substitute the term x + y ior z: we get 

\/x3y P{x, y,x + y). 

We see that the variables x and y occurring in the term x + y become bound variables after 
substitution. We say that there is a "capture of variables" . 

This is not what we intended to happen! To fix this problem, we recall that bound vari- 
ables are really place holders, so they can be renamed without changing anything. Therefore, 
we can rename the bound variables x and y in Vx3|/ P{x, y, z) to u and getting the state- 
ment \/u3v P{u, V, z) and now, the result of the substitution is 

VmSv v,x + y). 

Again, all this needs to be explained very carefuly but this can be done! 

Finally, here are the inference rules for the quantifiers, first stated in a natural deduction 
style and then in sequent style. It is assumed that we use two disjoint sets of variables for 
labeling premises {x,y, - ■ ■ ) and free variables {t,u,v, • • • ). As we will see, the V-introduction 
rule and the 3-elimination rule involve a crucial restriction on the occurrences of certain 
variables. Remember, variables are terms\ 

Definition 1.6.1 The inference rules for the quantifiers are 
\/ -introduction: 

V 

VtP 

Here, u must be a variable that does not occur free in any of the propositions in F or in 
VtP; the notation P[u/t] stands for the result of substituting u for all free occurrences of t 
in P. 

\/- elimination: 

F 

VtP 
P[r/t] 

Here r is an arbitrary term and it is assumed that bound variables in P have been renamed 
so that none of the variables in r are captured after substitution. 

3 -introduction: 

F 

P[r/t] 
3tP 
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As in V-elimination, r is an arbitrary term and the same proviso on bound variables in 
P applies. 

3 -elimination: 

r A,p[u/tY 

3tP C 

X 

C 

Here, u must be a variable that does not occur free in any of the propositions in A, 3tP, 
or C, and all premises P[u/i\ labeled x are discharged. 

In the above rules, F or A may be empty, P, C denote arbitrary propositions constructed 
from a first-order language, L and t is any variable. The system oi first- order classical logic, 
A/g^'^'^'"*"'^'^ is obtained by adding the above rules to the system of propositional classical 
logic A/g^'^'^'"*". The system of first-order intuitionistic logic, J\f7^'^'^'^^'^ is obtained by 
adding the above rules to the system of propositional intuitionistic logic J\fr^'^'^'-^. 



Using sequents, the quantifier rules in first-order logic are expressed as follows: 

Definition 1.6.2 The inference rules for the quantifiers in Gentzen- sequent style are 

T^PVa/t] ,^ . , V^NtP ,^ , 
V^MtP ^^-^"'"'^ r P\rlt\ ^^-'^^"^^ 

where in {\/-intro), u does not occur free in F or MtP; 

{3-intro) 3-ezm 

where in (3-e/im), u does not occur free in F, 3tP , or C . Again, t is any variable. 

The variable u is called the eigenvariable of the inference. The systems jV^^'^'^--'-'^'^ 
and AT^f '^'^'"^'^'^ are defined from the systems AfGf'^'^'-^ and NQf''^'^'^, respectively, by 
adding the above rules. 



When we say that a proposition, P, is provable from, F, wc mean that we can construct 
a proof tree whose conclusion is P and whose set of premises is F, in one of the systems 
jy/-^=»,A,v,±,v,3 ^^=^,A,v,±,v,3 Therefore, as in propositional logic, when we use the word 
"provable" unqualified, we mean provable in classical logic. Otherwise, we say intuitionisti- 
cally provable . 

A first look at the above rules shows that universal formulae, \/tP, behave somewhat 
like infinite conjunctions and that existential formulae, 3tP, behave somewhat like infinite 
disjunctions. 
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The V-introduction rule looks a little strange but the idea behind it is actually very simple: 
Since u is totally unconstrained, if P[u/t] is provable (from F), then intuitively P[u/t] holds 
of any arbitrary object, and so, the statement VtP should also be provable (from F). 

The meaning of the V-elimination is that if VtP is provable (from F), then P holds for 
all objects and so, in particular for the object denoted by the term r, i.e., P[T/t] should be 
provable (from F). 

The 3-introduction rule is dual to the V-elimination rule. If P[T/t] is provable (from F), 
this means that the object denoted by r satisfies P, so 3tP should be provable (this latter 
formula asserts the existence of some object satisfying P, and r is such an object). 

The 3-elimination rule is reminiscent of the V-elimination rule and is a little more tricky. 
It goes as follows: Suppose that we proved 3tP (from F). Moreover, suppose that for every 
possible case, P[u/t], we were able to prove C (from F). Then, as we have "exhausted" all 
possible cases and as we know from the provability of 3tP that some case must hold, we can 
conclude that C is provable (from F) without using P[u/t\ as a premise. 

Like the V-elimination rule, the 3-elimination rule is not very constructive. It allows 
making a conclusion (C) by considering alternatives without knowing which actually occurs. 

Remark: Anagolously to disjunction, in (first-order) intuitionistic logic, if an existential 
statement 3tP is provable (from F), then from any proof of 3tP, some term, r, can be 
extracted so that -P[r/t] is provable from F. Such a term, r, is called a witness. The witness 
property is not easy to prove. It follows from the fact that intuitionistic proofs have a normal 
form (see Section [TT7I) . However, no such property holds in classical logic (for instance, see 
the rational with a, b irrational example revisited below). 

Here is an example of a proof in the system 7V'^=^'^'^'"'"'^'^ (actually, in jV^"*'^'^'-*-'^'^) of the 
formula Vt(P A Q) ^ VtP A VtQ. 

Vt(P A Q)^ Vt(P A g)^' 



P[u/t] A Q[u/t] P[u/t] A Q[u/t] 
P[u/t] Q[u/t] 

ytp \/tQ 
ytp A vtg 



vt(p A g) ^ vtp A vtg 

In the above proof, m is a new variable, i.e., a variable that does not occur free in P or Q. 

The reader should show that WtP A WtQ =^ Vt(P A Q) is also provable in tv^"^'^'^'-*-'^'^. 
However, in general, one can't just replace V by 3 (or A by V) and still obtain provable 
statements. For example, 3tP A 3tQ =^ 3t{P A Q) is not provable at all! 

Here are some useful equivalences involving quantifiers. The first two are analogous to 
the de Morgan laws for A and V. 
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Proposition 1.6.3 The following equivalences are provable in classical first-order logic: 

^VtP = 3t^P 

^3tp = yt^p 

Vt(P AQ)= ytp A VtQ 
3t{P V g) = 3tP V 3tQ. 

In fact, the last three and 3t-iP =^ -NtP are provable intuitionistically. Moreover, the 
propositions 3t{PAQ) =^ 3tPA3tQ andWtPVWtQ =^ Vt(PVQ) are provable in intuitionistic 
first-order logic (and thus, also in classical first- order logic). 

Proof . Left as an exercise to the reader. □ 

Remark: We can illustrate, again, the fact that classical logic allows for non-constructive 

proofs by reexamining the example at the end of Section ll.3[ There, we proved that if a/2 
is rational, then a = \/2 and h = \pl are both irrational numbers such that is rational 

and if -\/2 is irrational then a = \/2 and h = \/2 are both irrational numbers such that 

is rational. By 3-introduction, we deduce that if \/2 is rational then there exist some 

\/2 

irrational numbers a, h so that is rational and if \/2 is irrational then there exist some 
irrational numbers a, h so that is rational. In classical logic, as P V -iP is provable, by 
V-elimination, we just proved that there exist some irrational numbers a and h so that a}' is 
rational. 

However, this argument does not give us explicitely numbers a and h with the required 

properties! It only tells us that such numbers must exist. Now, it turns out that \/2 
is indeed irrational (this follows from the Gel'fond-Schneider Theorem, a hard theorem in 
number theory). Furthermore, there are also simpler explicit solutions such as a = \/2 and 
h = log2 9, as the reader should check! 

We conclude this section by giving an example of a "wrong proof" . Here is an example 
in which the V-introduction rule is applied illegally, and thus, yields a statement which is 
actually false (not provable). In the incorrect "proof" below, P is an atomic predicate symbol 
taking two arguments (for example, "parent") and is a constant denoting zero: 

P{t,OY 

illegal step! 



VtP(t, 0) 



P{t,0) ^ VtP(t,0) 
Vt(P(t,0) ^ VtP(t,0)) 
P(0,0) ^ VtP(t,0) 



The problem is that the variable t occurs free in the premise P[t/t,0] = P{t,0) and 
therefore, the application of the V-introduction rule in the first step is illegal. However, 
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note that this premise is discharged in the second step and so, the apphcation of the V- 
introduction rule in the third step is legal. The (false) conclusion of this faulty proof is that 
P(0,0) =^ VtP(t,0) is provable. Indeed, there are plenty of properties such that the fact 
that the single instance, P(0, 0), holds does not imply that P{t, 0) holds for all t. 

Remark: The above example shows why it is desirable to have premises that are universally 
quantified. A premise of the form \/tP can be instantiated to P[u/t], using V-elimination, 
where u is a. brand new variable. Later on, it may be possible to use V-introduction without 
running into trouble with free occurrences of u in the premises. But we still have to be very 
careful when we use V-introduction or 3-elimination. 

Before concluding this section, let us give a few more examples of proofs using the rules 
for the quantifiers. First, let us prove that 

ytp = yuP[u/t], 

where u is any variable not free in \/tP and such that u is not captured during the substitution. 
This rule allows us to rename bound variables (under very mild conditions). We have the 
proofs 

(ytp)" 

yuP[u/t] 

a 

ytp =^ yuP[u/t] 

and 

{VuP[u/t])" 

ytp 

a 

yuP[u/t] ytP 

Now, we give a proof (intuitionistic) of 

3t{P^Q) ^ {ytP^Q), 
where t does not occur (free or bound) in Q. 
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(ytp)y 

{P[u/t] ^ QY P[u/t] 
{3t{P ^ Q)r Q 

X 

Q 

y 

ytp^Q 

z 

3t{P ^ Q) (VtP Q) 

In the above proof, -u is a new variable that does not occur in \/tP, or 3t{P =^ Q) 
The converse requires (RAA) and is a bit more comphcated. To conclude, we give a proof 
(intuitionistic) of 

(ViPvg) =^ vt(Pv(5), 

where t does not occur (free or bound) in Q. 

(itPY 

P[u/t] Qy 

P[u/t] V Q P[u/t] V Q 

(vtp V QY yt{p V Q) yt{p V Q) 

Vt(P V Q) 

z 

{ytp V Q) =^ yt{p V Q) 

In the above proof, u is a new variable that does not occur in ViP or Q. The converse 
requires (RAA). 

Several times in this Chapter, we have claimed that certain propositions are not provable 
in some logical system. What kind of reasoning do we use to validate such claims? In the 
next section, we briefly address this question as well as related ones. 

1.7 Decision Procedures, Proof Normalization, 
Counter- Examples, Theories, etc. 

In the previous sections, we saw how the rules of mathematical reasoning can be formalized 
in various natural deduction systems and we defined a precise notion of proof. We observed 
that finding a proof for a given proposition was not a simple matter, nor was it to acertain 
that a proposition is unprovable. Thus, it is natural to ask the following question: 

The Decision Problem: Is there a general procedure which takes any arbitrary proposi- 
tion, P, as input, always terminates in a finite number of steps, and tells us whether P is 
provable or not. 
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Clearly, it would be very nice if such a procedure existed, especially if it also produced a 
proof of P when P is provable. 

Unfortunately, for rich enough languages, such as first-order logic, it is impossible to 
find such a procedure. This deep result known as the undecidability of the decision prob- 
lem or Church's Theorem was proved by A. Church in 1936 (Actually, Church proved the 
undecidability of the validity problem, but by Godel's completeness Theorem, validity and 
provability are equivalent). 

Proving Church's Theorem is hard and a lot of work. One needs to develop a good deal 
of what is called the theory of computation. This involves defining models of computation 
such as Turing machines and proving other deeps results such as the undecidability of the 
halting problem and the undecidability of the Post Correspondence Problem, among other 
things. Some of this material is covered in CSE262, so be patient and your curiosity will be 
satisfied! 

So, our hopes to find a "universal theorem prover" are crushed. However, if we restrict 
ourselves to prepositional logic, classical or intuitionistic, it turns out that procedures solving 
the decision problem do exist and they even produce a proof of the input proposition when 
that proposition is provable. 

Unfortunately, proving that such procedures exist and are correct in the propositional case 
is rather difficult, especially for intuitionistic logic. The difficulties have a lot to do with our 
choice of a natural deduction system. Indeed, even for the system A/"^ (or MQ"^), provable 
propositions may have infinitely many proofs. This makes the search process impossible; 
when do we know how to stop, especially if a proposition is not provable! The problem is that 
proofs may contain redundancies (Gentzen said "detours"). A typical example of redundancy 
is an elimination immediately follows an introduction, as in the following example in which 
Vi denotes a deduction with conclusion T,x: A ^ B and T>2 denotes a deduction with 
conclusion T ^ A. 

T,x: A-^ B x>2 

r^A^B r^A 

V ^B 

Intuitively, it should be possible to construct a deduction for F — > S from the two deduc- 
tions T>i and T>2 without using at all the hypothesis x : A. This is indeed the case. If we look 
closely at the deduction T>i, from the shape of the inference rules, assumptions are never 
created, and the leaves must be labeled with expressions of the form F', A, x : A,y: C ^ C or 
F, A, x: A ^ A, where y x and either F = F' or F = F', y : C. We can form a new deduc- 
tion for F — > S as follows: in T>i, wherever a leaf of the form T, A,x: A ^ A occurs, replace 
it by the deduction obtained from V2 by adding A to the premise of each sequent in V2. 
Actually, one should be careful to first make a fresh copy of V2 by renaming all the variables 
so that clashes with variables in Vi are avoided. Finally, delete the assumption x: A from 
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the premise of every sequent in the resulting proof. The resulting deduction is obtained by 
a kind of substitution and may be denoted as T>i[D2/x], with some minor abuse of notation. 
Note that the assumptions x: A occurring in the leaves of the form F', A,x: A,y: C —>■ C 
were never used anyway. The step which consists in transforming the above redundant proof 
figure into the deduction Vi[D2/x] is called a reduction step or normalization step. 

The idea of proof normalization goes back to Gentzen (^20], 1935). Gentzen noted that 
(formal) proofs can contain redundancies, or "detours" , and that most complications in the 
analysis of proofs are due to these redundancies. Thus, Gentzen had the idea that the analysis 
of proofs would be simplified if it was possible to show that every proof can be converted to 
an equivalent irredundant proof, a proof in normal form. Gentzen proved a technical result 
to that effect, the "cut-elimination theorem" , for a sequent-calculus formulation of first-order 
logic [20]. Cut- free proofs are direct, in the sense that they never use auxiliary lemmas via 
the cut rule. 

Remark: It is important to note that Gentzen's result gives a particular algorithm to pro- 
duce a proof in normal form. Thus, we know that every proof can be reduced to some normal 
form using a specific strategy, but there may be more than one normal form, and certain 
normalization strategies may not terminate. 

About thirty years later, Prawitz ([35j, 1965) reconsidered the issue of proof normal- 
ization, but in the framework of natural deduction rather than the framework of sequent 
calculio Prawitz explained very clearly what redundancies are in systems of natural deduc- 
tion, and he proved that every proof can be reduced to a normal form. Furthermore, this 
normal form is unique. A few years later, Prawitz ([36], 1971) showed that in fact, every 
reduction sequence terminates, a property also called strong normalization. 

A remarkable connection between proof normalization and the notion of computation 
must also be mentioned. Curry (1958) made the remarkably insightful observation that 
certain typed combinators can be viewed as representations of proofs (in a Hilbert system) 
of certain propositions (See in Curry and Feys [T2] (1958), Chapter 9E, pages 312-315.) 
Building up on this observation, Howard (|i29j, 1969) described a general correspondence 
between propositions and types, proofs in natural deduction and certain typed A-terms, 
and proof normalization and /3-reduction. (The simply-tj^ed-A-calculus was invented by 
Church, 1940). This correspondence, usually referred to as the Curry /Howard isomorphism 
or formulae-as-types principle, is fundamental and very fruitful. 

The Curry/Howard isomorphism establishes a deep correspondence between the notion 
of proof and the notion of computation. Furthermore, and this is the deepest aspect of 
the Curry/Howard isomorphism, proof normalization corresponds to term reduction in the 
A-calculus associated with the proof system. To make the story short, the correspondence 
between proofs in intuitionistic logic and typed A-terms on one-hand and between proof 
normalization and /9-conversion on the other hand can be used to translate results about 

^This is somewhat ironical, since Gentzen began his investigations using a natural deduction system, but 
decided to switch to sequent calculi (known as Gentzen systems!) for technical reasons. 
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typed A-terms into results about proofs in intuitionistic logic. By the way, some aspects of 
the Curry/Howard isomorphism are covered in CIS500. 

In summary, using either some suitable intuitionistic sequent calculi and Gentzen's cut 
elimination theorem or some suitable typed A-calculi and (strong) normalization results 
about them, it is possible to prove that there is a decision procedure for propositional intu- 
itionistic logic. However, it can also be shown that the time-complexity of any such procedure 
is very high. Here, we are alluding to complexity theory, another active area of computer 
science. You will learn about some basic and fundamental aspects of this theory in CSE262 
when you learn about the two problems P and NP. 

Readers who wish to learn more about these topics can read my two survey papers Gallier 
[T7] (on the Correspondence Between Proofs and A- Terms) and Gallier |[16| (A Tutorial on 
Proof Systems and Typed A-Calculi), both available on the web site 

http: / /www. cis.upenn.edu/~jean/gbooks/logic. html 

and the excellent introduction to proof theory by Troelstra and Schwichtenberg ^T\ . 

Anybody who really wants to understand logic should of course take a look at Kleene 
pi] (the famous "I.M."), but this is not recommended to beginners! 

Let us return to the question of deciding whether a proposition is not provable. To 
simplify the discussion, let us restrict our attention to propositional classical logic. So far, 
we have presented a very proof-theoretic view of logic, that is, a view based on the notion 
of provability as opposed to a more semantic view of based on the notions of truth and 
models. A possible excuse for our bias is that, as Peter Andrews (from CMU) puts it, 
"truth is elusive" . Therefore, it is simpler to understand what truth is in terms of the more 
"mechanical" notion of provability. (Peter Andrews even gave the subtitle 

To Truth Through Proof 

to his logic book Andrews [T]!) 

However, mathematicians are not mechanical theorem provers (even if they prove lots of 
stuff)! Indeed, mathematicians almost always think of the objects they deal with (functions, 
curves, surfaces, groups, rings, etc.) as rather concrete objects (even if they may not seem 
concrete to the uninitiated) and not as abstract entities soleley characterized by arcane 
axioms. 

It is indeed natural and fruitful to try to interpret formal statements semantically. For 
propositional classical logic, this can be done quite easily if we interpret atomic propositional 
letters using the truth values true and false. Then, the crucial point that every provable 
proposition (say in MQ^''^'^''^ ) has the value true no matter how we assign truth values to 
the letters in our proposition. In this case, we say that P is valid. 

The fact that provable implies valid is called soundness or consistency of the proof system. 
The soundness of the proof system A/"^^'^'^'"*" is easy to prove. For this, given any sequent, 
r — >■ P, we prove that whenever all the propositions in F are assigned the value true, then P 
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evaluates to true. This is easy to do: check that this holds for the axioms and that whenever 
it holds for the premise (s) of an inference rule then it holds for the conclusion. 

We now have a method to show that a proposition, P, is not provable: Find some truth 
assignment that makes P false. 

Such an assignment falsifying P is called a counter-example. If P has a counter-example, 
then it can't be provable because if it were, then by soundness it would be true for all 
possible truth assigments. 

But now, another question comes up: If a proposition is not provable, can we always find 
a counter-example for it. Equivalently, is every valid proposition provahlel If every valid 
proposition is provable, we say that our proof system is complete (this is the completeness 
of our system). 

The system A/"^^'^'^'"*" is indeed complete. In fact, all the classical systems that we 
have discussed are sound and complete. Completeness is usually a lot harder to prove than 
soundness. For first-order classical logic, this is known as Gddel's completeness Theorem 
(1929). Again, we refer our readers to Gallier [IB] van Dalen |32] or or Huth and Ryan 
[30] for a thorough discussion of these matters. In the first-order case, one has to define 
first-order structures (or first-order models). 

What about intuitionistic logic? 

Well, one has to come up with a richer notion of semantics because it is no longer true 
that if a proposition is valid (in the sense of our two- valued semantics using true, false), 
then it is provable. Several semantics have been given for intuitionistic logic. In our opinion, 
the most natural is the notion of Kripke model. Then, again, soundness and completeness 
holds for intuitionistic proof systems, even in the first-order case (see van Dalen [12]). 

In summary, semantic models can be use to provide counter-examples of unprovable 
propositions. This is a quick method to establish that a proposition is not provable. 

The way we presented deduction trees and proof trees may have given our readers the 
impression that the set of premises, F, was just an auxiliary notion. Indeed, in all of our 
examples, F ends up being empty! However, nonempty F's are crucially needed if we want to 
develop theories about various kinds of structures and objects, such as the natural numbers, 
groups, rings, fields, trees, graphs, sets, etc. Indeed, we need to make definitions about the 
objects we want to study and we need to state some axioms asserting the main properties 
of these objects. We do this by putting these definitions and axioms in F. Actually, we have 
to allow F to be infinite but we still require that our deduction trees are finite; they can 
only use finitely many of the propositions in F. We are then interested in all propositions, 
P, such that A ^ P is provable, where A is any finite subset of F; the set of all such P's is 
called a theory. Of course we have the usual problem of consistency: If we are not careful, 
our theory may be inconsistent, i.e., it may consist of all propositions. 

Let us give two examples of theories. 
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Our first example is the theory of equality. Indeed, our readers may have noticed that 
we have avoided to deal with the equality relation. In practice, we can't do that. 

Given a language, L, with a given supply of constant, function and predicate symbols, 
the theory of equality consists of the following formulae taken as axioms: 

\/{x — x) 

Vxi • • • Vx„Vyi • • -yvniixi = yi A • • • A x„ = y„) =^ /(xi, . . . , x„) = f{yi, . . . , 
Vxi • • • VxnV?/i • • • V|/n[(a;i = A • • • A = ?/n) A P{xi, . . . , x„) =^ P{yi, yn)], 

for all function symbols (of n arguments) and all predicate symbols (of n arguments), in- 
cluding the equality predicate, =, itself. 

It is not immediately clear from the above axioms that = is reflexive and transitive but 
this can shown easily. 

Our second example is the first-order theory of the natural numbers known as Peano 's 
arithmetic. 

Here, we have the constant (zero), the unary function symbol S (for successor function; 
the intended meaning is S{n) — n + 1) and the binary function symbols -|- (for addition) 
and * (for multiplication). In addition to the axioms for the theory of equality we have the 
following axioms: 

yx^{s{x) = 0) 

yxyy{S{x)^S{y)^x^y) 

VxVy(x + — x) 

yxyy{x + S{y)^S{x + y)) 

\/x\/y{x * = 0) 

WxWy{x * S{y) = x * y + x) 

[A{0) Ayx{A{x) =^ A{S{x)))] =^ ynA{n), 

where A is any first-order formula with one free variable. This last axiom is the induction 
axiom. Observe how + and * are defined recursively in terms of and S and that there are 
infinitely many induction axioms (countably many). 

Many properties that hold for the natural numbers (i.e., are true when the symbols 
0, S, +, * have their usual interpretation and all variables range over the natural numbers) 
can be proved in this theory (Peano's arithmetic), but not all! This is another very famous 
result of Godel known as Gddel's incompleteness Theorem (1931). However, the topic of 
incompleteness is definitely oustside the scope of this course, so we will not say anymore about 
it. Another very interesting theory is set theory. There are a number of axiomatizations of 
set theory and we will discuss one of them (ZF) very briefly in the next section. 

We close this section by repeating something we said ealier: There isn't just one logic 
but instead, many logics. In addition to classical and intuitionistic logic (propositional and 
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first-order), there are: modal logics, higher-order logics and linear logic, a logic due to Jean- 
Yves Girard, attempting to unify classical and intuitionistic logic (among other goals). An 
excellent introduction to these logics can be found in Troelstra and Schwichtenberg [51]. We 
warn our readers that most presentations of linear logic are (very) difficult to follow. This is 
definitely true of Girard's seminal paper [22] . A more approachable version can be found in 
Girard, Lafont and Taylor [21] , but most readers will still wonder what hit them when they 
attempt to read it. 

In computer science, there is also dynamic logic, used to prove properties of programs 
and temporal logic and its variants (originally invented by A. Pnueli), to prove properties of 
real-time systems. So, logic is alive and well! Also, take a look at CSE482! 

1.8 Basics Concepts of Set Theory 

Having learned some fundamental notions of logic, it is now a good place before proceeding 
to more interesting things, such as functions and relations, to go through a very quick review 
of some basic concepts of set theory. This section will take the very "naive" point of view 
that a set is a collection of objects, the collection being regarded as a single object. Having 
first-order logic at our disposal, we could formalize set theory very rigorously in terms of 
axioms. This was done by Zermelo first (1908) and in a more satisfactory form by Zermelo 
and Frankel in 1921, in a theory known as the "Zermelo-Frankel" (ZF) axioms. Another 
axiomatization was given by John von Neumann in 1925 and later improved by Bernays in 
1937. A modification of Bernay's axioms was used by Kurt Godel in 1940. This approach 
is now known as "von Neumann-Bernays" (VNB) or "Godel-Bernays" (GB) set theory. 
There are many books that give an axiomatic presentation of set theory. Among them, we 
recommend Enderton [14j, which we find remarkably clear and elegant, Suppes [30] (a little 
more advanced) and Halmos [27], a classic (at a more elementary level). 

However, it must be said that set theory was first created by Georg Cantor (1845-1918) 
between 1871 and 1879. However, Cantor's work was not unanimously well received by 
all mathematicians. Cantor regarded infinite objects as objects to be treated in much the 
same way as finite sets, a point of view that was shocking to a number of very prominent 
mathematicians who bitterly attacked him (among them, the powerful Kronecker). Also, 
it turns out that some paradoxes in set theory popped up in the early 1900, in particular, 
Russell's paradox. Russell's paradox (found by Russell in 1902) has to to with the 

"set of all sets that are not members of themselves" 

which we denote by 

R = {x \ X ^ x}. 

(In general, the notation {x \ P} stand for the set of all objects satisfying the property P.) 

Now, classically, either R & R or R ^ R. However, if i? G -R, then the definition of R 
says that R ^ R; if R ^ R, then again, the definition of R says that R & R\ 
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So, we have a contradiction and the existence of such a set is a paradox. The problem 
is that we are allowing a property (here, P{x) = x ^ x), which is "too wild" and circular 
in nature. As we will see, the way out, as found by Zcrmclo, is to place a restriction on the 
property P and to also make sure that P picks out elements from some already given set 
(see the Subset Axioms below). 

The apparition of these paradoxes prompted mathematicians, with Hilbert among its 
leaders, to put set theory on firmer grounds. This was achieved by Zermelo, Frankel, von 
Neumann, Bernays and Godel, to only name the major players. 

In what follows, we are assuming that we are working in classical logic. We will introduce 
various operations on sets using defintion involving the logical connectives A, V, -i, V and 
3. In order to ensure the existence of some of these sets requires some of the axioms of set 
theory, but we will be rather casual about that. 

Given a set. A, we write that some object, a, is an element of (belongs to) the set A as 

a e A 

and that a is not an element of A (does not belong to A) as 

a ^ A. 

When are two sets A and B equal? This corresponds to the first axiom of set theory, 
called 

Extensionality Axiom 

Two sets A and B are equal iff they have exactly the same elements, that is 

yx{x e A^ X e B) A yx{x e B ^ X e A). 

The above says: Every element of A is an clement of B and conversely. 

There is a special set having no elements at all, the empty set, denoted 0. This is the 
Empty Set Axiom 

There is a set having no members. This set is denoted 0. 

Remark: Beginners often wonder whether there is more than one empty set. For example, 
is the empty set of professors distinct from the empty set of potatoes? 

The answer is, by the extensionality axiom, there is only one empty set! 

Given any two objects a and b, wc can form the set {a, b} containing exactly these two 
objects. Amazingly enough, this must also be an axiom: 

Pairing Axiom 
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Given any two objects a and b (think sets), there is a set, {a, b}, having as members just 
a and b. 

Observe that if a and b are identical, then we have the set {a, a}, which is denoted by 
{a} and is called a singleton set (this set has a as its only element). 

To form bigger sets, we use the union operation. This too requires an axiom. 

Union Axiom (Version 1) 

For any two sets A and B, there is a set, AU B, called the union of A and B defined by 

xeAuB iff {xeA)y{xeB). 

This reads, x is a member of ^4 U S if either x belongs to ^4 or x belongs to B (or both) . We 
also write 

AU B = {x \ X e A or x e B}. 

Using the union operation, we can form bigger sets by taking unions with singletons. For 
example, we can form 

{a, b, c} = {a, b} U {c}. 

Remcirk: We can systematically construct bigger and bigger sets by the following method: 
Given any set. A, let 

A+ = AU{A}. 

If we start from the empty set, we obtain sets that can be used to define the natural numbers 
and the + operation corresponds to the successor function on the natural numbers, i.e., 
n I— > n + 1. 

Another operation is the power set formation. It is indeed a "powerful" operation, in 
the sense that it allows us to form very big sets. For this, it is helpful to define the notion 
of inclusion between sets. Given any two sets, A and B, we say that A is a subset of B (or 
that A is included in B), denoted A C B, iS every element of A is also an element of B, i.e. 

yx{x e A^ X e B). 

We say that A is a proper subset of B iS A C B and A B. This implies that that there is 
some b E B with b ^ A. We usually write A C B. 

Observe that the equality of two sets can be expressed by 

A = B iff ACB and B C A. 

Power Set Axiom 

Given any set. A, there is a set, V{A), (also denoted 2"^) called the power set of A whose 
members are exactly the subsets of A, i.e.. 



X e V{A) iff X QA. 
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For example, HA— {a, b, c}, then 

V{A) = {0, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c}}, 

a set containing 8 elements. Note that the empty set and A itself are always members of 
V{A). 

Remcirk: If A has n elements, it is not hard to show that V{A) has 2" elements. For this 
reason, many people, including me, prefer the notation 2^ for the power set of A. 

At this stage, we would like to define intersection and complementation. For this, given 

any set, A, and given a property, P, (specified by a first-order formula) we need to be able 
to define the subset of A consisting of those elements satisfying P. This subset is denoted 
by 

{xeA \ P}. 

Unfortunately, there are problems with this construction. If the formula, P, is somehow a 
circular definition and refers to the subset that we are trying to define, then some paradoxes 
may arise! 

The way out is to place a restriction on the formula used to define our subsets, and 
this leads to the subset axioms, first formulated by Zermelo. These axioms are also called 
comprehension axioms or axioms of separation. 

Subset Axioms 

For every first-order formula, P, we have the axiom: 

yA3xyx{x eX iff {xeA)AP), 

where P does not contain X as a free variable. (However, P may contain x free.) 

The subset axiom says that for every set. A, there is a set, X, consisting exactly of those 
elements of A so that P holds. For short, we usually write 

X ^ {x e A \ P}. 

As an example, consider the formula 

P{B,x) ^xeB. 

Then, the subset axiom says 

yA3xyx(x eAaxeb), 

which means that X is the set of elements that belong both to A and B. This is called the 
intersection of A and B, denoted hy An B. Note that 

AnB = {x\xeA and x e B}. 
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We can also define the relative complement of B in A, denoted A — B, given by the 
formula P{x, B) — x ^ B,so that 

A - B = {x\x e A and x ^ B]. 

In particular, if A is any given set and B is any subset of A, the set A — i? is also denoted 
B and is called the complement of B. Because A, V and -i satisfy the de Morgan laws 
(remember, we are dealing with classical logic), for any set X, the operations of union, 
intersection and complementation on subsets of X satisfy various identities, in particular 
the de Morgan laws 

A\JB = ~AnB 

1 = A, 

and various associativity, commutativity and distributivity laws. 

So far, the union axiom only applies to two sets but later on we will need to form infinite 
unions. Thus, it is necessary to generalize our union axiom as follows: 

Union Axiom (Final Version) 

Given any set X (think of X as a set of sets), there is a set, {jX, defined so that 

xe\JX iff 3B{B e X Ax e B). 

This says that (J X consists of all elements that belong to some member of X. 
If we take X — {A, B}, where A and B are two sets, we see that 

[j{A,B} = AUB, 

and so, our final version of the union axiom subsumes our previous union axiom which we 
now discard in favor of the more general version. 

Observe that 

\J{A}=A, \J{Ai,...,An} = AiU---UAn. 

and in particular, (J = 0. 

Using the subset axiom, we can also define infinite intersections. For every nonempty 
set, X, there is a set, P|X, defined by 

xe[]x iff yB(BeX^xeB). 

The existence of P| X is justified as follows: Since X is nonempty, it contains some set, 
A; let 

P{X, x) = WB{B eX ^xeB). 
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Then, the subset axiom asserts the existence of a set Y so that for every x, 

X eY iff X e A and P{X, x) 

which is equivalent to 

xeY iff P{X,x). 
Therefore, the set Y is our desired set, f]X. 
Observe that 

f|{A B} = AnB, f|{Ai, . . . , A„} = n ■ ■ • n A,. 

Note that H ^ is not defined. Intuitively, it would have to be the set of all sets, but such a 
set does not exist, as we now show. This is basically a version of Russell's paradox. 

Theorem 1.8.1 (Russell) There is no set of all sets, i.e., there is no set to which every 
other set belongs. 

Proof. Let A be any set. We construct a set, B, that does not belong to A. If the set of all 
sets existed, then we could produce a set that does not belong to it, a contradiction. Let 

B = {a e A \ a ^ a}. 

We claim that B ^ A. We proceed by contradiction, so assume B e A. However, by the 
definition of S, we have 

B eB iff B eA and B ^ B. 
Since B e A, the above is equivalent to 

B eB iff B ^B, 

which is a contradiction. Therefore, B ^ A and we deduce that there is no set of all sets. □ 
Reiiicirks: 

(1) We should justify why the equivalence B&BiSB^B is a contradiction. What we 
mean by "a contradiction" is that if the above equivalence holds, then we can derive ± 
(falsity) and thus, all propositions become provable. This is because we can show that 
for any proposition, P, if P = -iP is provable, then -i(P = ^P) is also provable. We 
leave the proof of this fact as an easy exercise for the reader. By the way, this holds 
classically as well as intuitionistically. 
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(2) We said that in the subset axiom, the variable X is not allowed to occur free in 
P. A slight modification of Russell's paradox shows that allowing X to be free in 
P lead to paradoxical sets. For example, pick A to be any nonempty set and set 
P{X,x) = X ^ X. Then, look at the (alleged) set 

X^{xeA\x^X}. 

As an exercise, the reader should show that X is empty iff X is nonempty! 



This is as far as we can go with the elementary notions of set theory that we have 
introduced so far. In order to proceed further, we need to define relations and functions, 
which is the object of the next Chapter. 

The reader may also wonder why we have not yet discussed infinite sets. This is because 
we don't know how to show that they exist! Again, perhaps surprinsingly, this takes another 
axiom, the axiom of infinity. We also have to define when a set is infinite. However, we will 
not go into this right now. Instead, we will accept that the set of natural numbers, N, exists 
and is infinite. Once, we have the notion of a function, we will be able to show that other 
sets are infinite by comparing their "size" with that of N (This is the purpose of cardinal 
numbers, but this would lead us too far afield). 

Remark: In an axiomatic presentation of set theory, the natural numbers can be defined 
from the empty set using the operation A i— > A~^ = AU {A} introduced just after the union 
axiom. The idea due to von Neumann is that 

= 

1 = o+ = {0} = {O} 

2 = 1+ = {0,{0}} = {O,1} 

3 = 2+ = {0,{0},{0,{0}}} = {O,l,2} 

n + 1 = n+ = {0,1,2,..., n} 



However, the above subsumes induction! Thus, we have to proceed in a different way to 
avoid circularities. 

Definition 1.8.2 We say that a set, X, is inductive iff 

(1) $eX; 

(2) For every A E X, we have G X. 
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Axiom of Infinity 

There is some inductive set. 

Having done this, we make the 

Definition 1.8.3 A natural number is a set that belongs to every inductive set. 

Using the subset axioms, we can show that there is a set whose members are exactly 
the natural numbers. The argument is very similar to the one used to prove that arbitrary 
intersections exist. By the Axiom of infinity, there is some inductive set, say A. Now consider 
the property, P{x), which asserts that x belongs to every inductive set. By the subset axioms 
applied to P, there is a set, N, such that 

X eN iff X e A and P{x) 

and since A is inductive and P says that x belongs to every inductive set, the above is 
equivalent to 

xeN iff P{x), 

that is, X e N iff X belongs to every inductive set. Therefore, the set of all natural numbers, 
N, does exist. The set N is also denoted lo. We can now easily show 

Theorem 1.8.4 The set N is inductive and it is a subset of every inductive set. 

Proof. Recall that belongs to every inductive set; so, is a natural number (0). As N is 
the set of natural numbers, (= 0) belongs to N. Secondly, if n G N, this means that n 
belongs to every inductive set (n is a natural number), which implies that n+ = n+1 belongs 
to every inductive set, which means that n + 1 is a natural number, i.e., n + 1 e N. Since 
N is the set of natural numbers and since every natural number belongs to every inductive 
set, we conclude that N is a subset of every inductive set. □ 

It would be tempting to view N as the intersection of the family of inductive sets, but 
unfortunately this family is not a set; it is too "big" to be a set. 

As a consequence of the above fact, we obtain the 

Induction Principle for N: Any inductive subset of N is equal to N itself. 

Now, in our setting, = and n+ = n + 1, so the above principle can be restated as 
follows: 

Induction Principle for N (Version 2): For any subset, 5* C N, if G 5* and n + 1 E S 
whenever n E S, then 5" = N. 

We will see how to rephrase this induction principle a little more conveniently in terms 
of the notion of function in the next chapter. 



Remarks: 
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1. We still don't know what an infinite set is or, for that matter, that N is infinite! This 
will be shown in the next Chapter (see Corollary 12. 9. 71) . 

2. Zermelo-Frankel set theory (+ Choice) has three more axioms that we did not discuss: 
The Axiom of Choice, the Replacement Axioms and the Regularity Axiom. For our 
purposes, only the Axiom of Choice will be needed and we will introduce it in Chapter 
[2j Let us just say that the Replacement Axioms are needed to deal with ordinals and 
cardinals and that the Regularity Axiom is needed to show that every set is grounded. 
For more about these axioms, see Enderton [14], Chapter 7. The Regularity Axiom 
also implies that no set can be a member of itself, an eventuality that is not ruled out 
by our current set of axioms! 



Chapter 2 



Relations, Functions, Partial 
Functions 

2.1 What is a Function? 

We use functions all the time in Mathematics and in Computer Science. But, what exactly 
is a function? 

Roughly speaking, a function, /, is a rule or mechanism, which takes input values in 
some input domain, say X, and produces output values in some output domain, say Y, in 
such a way that to each input x E X corresponds a unique output value y E Y, denoted 
f{x). We usually write y = f{x), or better, x i— * f{x). 

Often, functions are defined by some sort of closed expression (a formula), but not always. 
For example, the formula 

y^2x 

defines a function. Here, we can take both the input and output domain to be R, the set of 

real numbers. Instead, we could have taken N, the set of natural numbers; this gives us a 
different function. In the above example, 2x makes sense for all input x, whether the input 
domain is N or R, so our formula yields a function defined for all of its input values. 

Now, look at the function defined by the formula 

X 

If the input and output domains are both M, again this function is well-defined. However, 
what if we assume that the input and output domains are both N? This time, we have a 
problem when x is odd. For example, | is not an integer, so our function is not defined for 
all of its input values. It is a partial function. Observe that this function is defined for the 
set of even natural numbers (sometimes denoted 2N) and this set is called the domain (of 
definition) of /. If we enlarge the output domain to be Q, the set of rational numbers, then 
our function is defined for all inputs. 



61 



62 



CHAPTER 2. RELATIONS, FUNCTIONS, PARTIAL FUNCTIONS 



Another example of a partial function is given by 

_ x + 1 
^ — 3x + 2' 

assuming that both the input and output domains are M. Observe that for x = 1 and x = 2, 
the denominator vanishes, so we get the undefined fractions | and |. The function "blows 
up" ior X — 1 and x — 2, its value is "infinity" (= oo), which is not an element of R. So, the 
domain of / is IR - {1, 2}. 

In summary, functions need not be defined for all of their input values and we need to 
pay close attention to both the input and the ouput domain of our functions. 

The following example illustrates another difficulty: Consider the function given by 

y = Vx. 

If we assume that the input domain is R and that the output domain is M"*" = {x e R | x > 0}, 
then this function is not defined for negative values of x. To fix this problem, we can extend 
the output domain to be C, the complex numbers. Then we can make sense of ^/x when 
a; < 0. However, a new problem comes up: Every negative number, x, has two complex 
square roots, — and (where i is "the" square root of —1). Which of the two 

should we pick? 

In this case, we could systematically pick +i\^—x but what if we extend the input domain 
to be C. Then, it is not clear which of the two complex roots should be picked, as there is 
no obvious total order on C. We can treat / as a multi-valued function, that is, a function 
that may return several possible outputs for a given input value. 

Experience shows that it is akward to deal with multi-valued functions and that it is 
best to treat them as relations (or to change the output domain to be a power set, which is 
equivalent to view the function as a relation) . 

Let us give one more example showing that it is not always easy to make sure that a 
formula is a proper definition of a function. Consider the function from M to R given by 

n=l 

Here, n\ is the function factorial, defined by 

n! = n-(n-l)---2-l. 

How do we make sense of this infinite expression? Well, that's where analysis comes in, 
with the notion of limit of a series, etc. It turns out that f{x) is the exponential function 
f{x) = e^. Actually, is even defined when x is a complex number or even a square matrix 
(with real or complex entries)! Don't panic, we will not use such functions in this course. 
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Another issue comes up, that is, the notion of computability. In all of our examples, 
and for most functions we will ever need to compute, it is clear that it is possible to give 
a mechanical procedure, i.e., a computer program which computes our functions (even if it 
hard to write such a program or if such a program takes a very long time to compute the 
output from the input). 

Unfortunately, there are functions which, although well-defined mathematically, are not 
computable! For an example, let us go back to first-order logic and the notion of provable 
proposition. Given a finite (or countably infinite) alphabet of function, predicate, constant 
symbols, and a countable supply of variables, it is quite clear that the set JF of all propositions 
built up from these symbols and variables can be enumerated systematically. We can define 
the function, Prov, with input domain JF and output domain {0, 1}, so that, for every 
proposition P G JF, 



Mathematically, for every proposition, P G JF, either P is provable or it is not, so this 
function makes sense. However, by Church's Theorem (see Section [L71) . we know that there 
is no computer program that will terminate for all input propositions and give an answer in a 
finite number of steps! So, although the function Prov makes sense as an abstract function, 
it is not computable. Is this a paradox? No, if we are careful when defining a function 
not to incorporate in the definition any notion of computability and instead to take a more 
abstract and, in some some sense naive view of a function as some kind of input/output 
process given by pairs (input value, output value) (without worrying about the way the 
output is "computed" from the input). A rigorous way to proceed is to use the notion of 
ordered pair and of graph of a function. Before we do so, let us point out some facts about 
functions that were revealed by our examples: 

1. In order to define a function, in addition to defining its input/output behavior, it is 
also important to specify what is its input domain and its output domain. 

2. Some functions may not be defined for all of their input values; a function can be a 
partial function. 

3. The input /output behavior of a function can be defined by a set of ordered pairs. As 
we will see next, this is the graph of the function. 

We are now going to formalize the notion of function (possibly partial) using the concept 
of ordered pair. 




1 if P is provable (classically) 

if P is not provable (classically). 
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2.2 Ordered Pairs, Cartesian Products, Relations, 
Functions, Partial Functions 

Given two sets, A and B, one of the basic constructions of set theory is the formation of an 
ordered pair, {a,b), where a & A and h E B. Sometimes, we also write (a, 6) for an ordered 
pair. The main property of ordered pairs is that if (ai,6i) and (02,62) are ordered pairs, 
where ai, 02 G A and 61, 62 G B, then 

(ai,6i) = (02,62) iff oi = 02 and hi = 62. 

Observe that this property imphes that, 

{a,h) ^ {b,a), 

unless a — b. Thus, the ordered pair, (a, 6), is not a notational variant for the set {a,b}; 
implicit to the notion of ordered pair is the fact that there is an order (even though we 
have not yet defined this notion yet!) among the elements of the pair. Indeed, in (a, 6), the 
clement a comes first and b comes second. Accordingly, given an ordered pair, p = {a,b), 
we will denote a by pri{p) and b by pr2{p) {first an second projection or first and second 
coordinate) . 

Remark: Readers who like set theory will be happy to hear that an ordered pair, (a, b), can 
be defined as the set {{a}, {a,b}}. This definition is due to Kuratowski, 1921. An earlier 
(more complicated) definition given by N. Wiener in 1914 is {{{a}, 0}, {{6}}}. 

Now, from set theory, it can be shown that given two sets, A and B, the set of all ordered 
pairs (a, b), with a & A and 6 e S, is a set denoted A x B and called the Cartesian product 
of A and B (in that order). By convention, we agree that $xB — Ax$ — $. To simplify 
the terminology, we often say pair for ordered pair, with the understanding that pairs are 
always ordered (otherwise, we should say set). 

Of course, given three sets. A, B, C, we can form {Ax B) x C and we call its elements 
(ordered) triples (or triplets). To simplify the notation, we write (a, b, c) instead of ((a, b), c). 
More generally, given n sets Ai, . . . , (n > 2), we define the set of n-tuples, 
Ai X ^42 X • • • X An, as (• • • {{Ai x A2) x A^) x • • • ) x A^. An element of x ^2 x • • • x 
is denoted by (ai, . . . , a^) (an n-tuple). We agree that when n = 1, we just have Ai and a 
1-tuple is just an element of A\. 

We now have all we need to define relations. 

Definition 2.2.1 Given two sets, A and B, a (binary) relation, R, between A and B is any 
subset RQ Ax B of ordered pairs from Ax B. When (a, b) E R, we also write aRb and we 
say that a and b are related by R. The set 



dom{R) = {a e A\3b E B, {a, b) E R] 
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is called the domain of R and the set 

range{R) = {b e B \ 3a e A, (a, b) G R} 

is called the range of R. Note that dom{R) C A and range{R) C B. When A = B,we often 
say that R is a (binary) relation over A. 

Among all relations between A and B, we mention three relations that play a special 
role: 

1. i? = 0, the empty relation. Note that dom{^) = range{^) = 0. This is not a very 
exciting relation! 

2. When A = B, we have the identity relation, 

id-A = {(a, a) \ a E A}. 

The identity relation relates every element to itself, and that's it! Note that 
(iom(id^) = range{idA) = A. 

3. The relation A x B itself. This relation relates every element of A to every element of 
B. Note that dom{A x B) = A and range{A x B) = B. 

Relations can be represented graphically by pictures often called graphs. (Beware, the 
term "graph" is very much overloaded. Later on, we will define what a graph is.) We depict 
the elements of both sets A and B as points (perhaps with different colors) and we indicate 
that a E A and b E B are related (i.e., {a,b) e R) by drawing an oriented edge (an arrow) 
starting from a (its source) and ending in b (its target). Here is an example: 



05 o 




Figure 2.1: A binary relation, R 

In Figure [2711 A = {01,02,03,04,05} and B = {&i, &2, ^Sj ^4}- Observe that 05 is not 
related to any element of B, 63 is not related to any element of A and that some elements 
of A, namely, Oi, 03, 04, are related some several elements of B. 
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Now, given a relation, R C Ax B , some element a G A may be related to several distinct 
elements b E B. If so, R does not correspond to our notion of a function, because we want 
our functions to be single-valued. So, we impose a natural condition on relations to get 
relations that correspond to functions. 

Definition 2.2.2 We say that a relation, R, between two sets A and B is functional if for 
every a E A, there is at most one b E B so that (a, b) G R. Equivalently, R is functional if 
for all a G -B and all 6i, &2 ^ B, if (a, bi) G R and (a, 62) € R, then 61 = 62- 

The picture in Figure 12.21 shows an example of a functional relation. 

05 o 
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Figure 2.2: A functional relation G 

Using Definition 12. 2. 2[ we can give a rigorous definition of a function (partial or not). 

Definition 2.2.3 A partial function, /, is a triple, / = {A,G,B), where A is a set called 
the input domain of /, i? is a set called the output domain of f (sometimes codomain of f) 
and G Ax B is a. functional relation called the graph of f; we let graph{f) = G. We write 
f : A ^ B to indicate that A is the input domain of / and that B is the codomain of / 
and we let dom{f) = dom{G) and range{f) = range{G). For every a G dom{f), the unique 
element, b E B, so that {a,b) G graph{f) is denoted by /(a) (so, b = /(a)). Often, we say 
that b = f{a) is the image of a by f. The range of / is also called the image of f and is 
denoted Q'(/). If dom{f ) = A, we say that / is a total function, for short, a function with 
domain A. 

Remarks: 

1. If / = {A, G, B) is a partial function and b = f{a) for some a G dom{f), we say that 
/ maps a to b; we may write / : a \—>- b. For any b E B, the set 



{aEA\ f{a) = b} 
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is denoted f^^{b) and called the inverse image or preimage ofb by f. (It is also called 
the fibre of f above b. We will explain this peculiar language later on.) Note that 
f~^{b) 7^ iff 6 is in the image (range) of /. Often, a function, partial or not, is called 
a map. 

2. Note that Definition 12.2.31 allows A = 0. In this case, we must have (7 = and, 
technically, (0, 0, B) is total function! It is the empty function from to B. 

3. When a partial function is a total function, we don't call it a "partial total function", 
but simply a "function". The usual pratice is that the term "function" refers to a 
total function. However, sometimes, we say "total function" to stress that a function 
is indeed defined on all of its input domain. 

4. Note that if a partial function / = {A, G, B) is not a total function, then dom{f) ^ A 
and for all a G A — dom{f), there is no 6 G -B so that {a,b) G graph{f). This 
corresponds to the intuitive fact that / does not produce any output for any value not 
in its domain of definition. We can imagine that / "blows up" for this input (as in the 
situation where the denominator of a fraction is 0) or that the program computing / 
loops indefinitely for that input. 

5. If / = {A, G, B) is a total function and A 7^ 0, then 7^ 0. 

6. For any set, A, the identity relation, id^, is actually a function id^: A —>■ A. 

7. Given any two sets, A and B, the rules (a, 6) a = pri{{a,b)) and {a,b) 1-^6 = 
pr2{{a,b)) make pri and pr2 into functions pri: A x B ^ A and pr2: A x B ^ B 
called the first and second projections. 

8. A function, f: A ^ B, is sometimes denoted A — > B. Some authors use a different 
kind of arrow to indicate that / is partial, for example, a dotted or dashed arrow. We 
will not go that far! 

9. The set of all functions, f: A ^ B, is denoted by B^. If A and B are finite, A has m 
elements and B has n elements, it is easy to prove that B^ has ra™ elements. 

The reader might wonder why, in the definition of a (total) function, f: A B, we do 
not require B = Qf, since we require that dom(/) = A. 

The reason has to do with experience and convenience. It turns out that in most cases, 
we know what the domain of a function is, but it may be very hard to determine exactly 
what its image is. Thus, it is more convenient to be flexible about the codomain. As long 
as we know that / maps into B, we are satisfied. 

For example, consider functions, /: M ^ M^, from the real line into the plane. The 
image of such a function is a curve in the plane M^. Actually, to really get "decent" curves 
we need to impose some reasonable conditions on /, for example, to be differentiable. Even 
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Figure 2.3: Lemniscate of Bernoulli 



continuity may yield very strange curves (see Section [2. lOp . But even for a very well behaved 
function, /, it may be very hard to figure out what the image of / is. Consider the function, 
t {x{t),y{y)), given by 



The curve which is the image of this function, shown in Figure 12. 3[ is called the "lemnis- 
cate of Bernoulli" . 

Observe that this curve has a self-intersection at the origin, which is not so obvious at 
first glance. 

2.3 Induction Principle on N 

Now that we have the notion of function, we can restate the induction principle (Version 
2) stated at the send of Section 11.81 to make it more flexible. We define a property of the 
natural numbers as any function, P: N — > {true, false}. The idea is that P{n) holds iff 
P{n) = true, else P{n) = false. Then, we have the following principle: 

Principle of Induction for N (Version 3). 

Let P be any property of the natural numbers. In order to prove that P{n) holds for all 
n G N, it is enough to prove that 



x{t) 



yit) 



t(i+f) 
i + t^ 

i + t* 



(1) P(0) holds and 



(2) For every n G N, the implication P{n) =^ P{n + 1) holds. 
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As a formula, (1) and (2) can be written 

[P(0) A (Vn G n){P{n) P{n + 1))] ^ (Vn G N)P(n). 

Step (1) is usually called the basis or base step of the induction and step (2) is called 
the induction step. In step (2), P{n) is called the induction hypothesis. That the above 
induction principle is valid is given by the 

Proposition 2.3.1 The Principle of Induction stated above is valid. 
Proof. Let 

^ = {n e N I P{n) = true}. 

By the induction principle (Version 2) stated at the send of Section [L8l it is enough to prove 
that S is inductive, because then = N and we are done. 

Since -P(O) hold, we have G S". Now, if n G S', i.e., if P(n) holds, since P{n) =^ P{n + 1) 
holds for every n, we deduce that P{n + 1) holds, that is, n + 1 G S. Therefore, S is inductive 
as claimed and this finishes the proof. □ 

Induction is a very valuable tool for proving properties of the natural numbers and we 
will make extensive use of it. We will also see other more powerful induction principles. Let 
us give just one example illustrating how it is used. 

Claim: 

l + 3 + 5 + --- + 2n + l = (n + l)2, 

where n G N. 

For the basis of the induction, where n = 0, we get 1 = 1^, so the base step holds. 
For the induction step, for any n G N, assume that 

l + 3 + 5 + -- - + 2n + l = (n + 1)^. 

Consider l + 3 + 5 + -- - + 2n+l + 2(n + l) + l = l + 3 + 5 + -- - + 2n + l + 2n + 3. Then, 
using the induction hypotesis, we have 

l + 3 + 5 + -- - + 2n + l + 2n + 3 = {n + lf + 2n + 'i 

= + 2n + l + 2n + 3 = n'^ + An + A 
= (n + 2)2. 

Therefore, the induction step holds and this completes the proof by induction. □ 
A useful way to produce new relations or functions is to compose them. 
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2.4 Composition of Relations and Functions 

We begin with the definition of the composition of relations. 

Definition 2.4.1 Given two relations, R O A x B and S B x C, the composition of R 
and S, denoted Ro S, is the relation between A and C defined by 

RoS = {{a,c) e AxC \3be B, {a,b) e R and {b,c) e S}. 

One should check that for any relation R O Ax B, we have id^ oR = R and i?o id^ = R. 
If R and S are the graphs of functions, possibly partial, is Ro S the graph of some function? 
The answer is yes, as shown in the following 

Proposition 2.4.2 Let R C A x B and S C B x C be two relations. 

(a) If R and S are both functional relations, then R o S is also a functional relation. 
Consequently, Ro S is the graph of some partial function. 

(h) If dom{R) = A and dom{S) = B, then dom{R o S) = A. 

(c) If R is the graph of a (total) function from A to B and S is the graph of a (total) 
function from B to C , then Ro S is the graph of a (total) function from A to C . 

Proof . (a) Assume that (a, Ci) E Ro S and (a, C2) G i? o S. By definition oi R o S, there 
exist 61 , 62 G -B so that 

(a, 61) G R, (61, Ci) G S, 
{a,b2)eR, (62,02) G 5. 

As R is functional, (a, 61) G R and (0,62) G R implies bi = 62- Let b = bi = 62, so that 
(61, Ci) = {b, Ci) and (62, C2) = {b, C2). But, S is also functional, so {b, Ci) G S and {b, C2) G S 
implies that Ci = C2, which proves that Ro S is functional. 

(b) Pick any a G A. The fact that dom{R) = A means that there is some b E B so that 
(a, b) E R. As S is also functional, there is some c G C so that {b, c) G S. Then, by the 
definition of Ro S, we see that (a, c) E Ro S. Since the argument holds for any a E A, we 
deduce that dom{R o S) = A. 

(c) If R and S are the graphs of partial functions, then this means that they are functional 
and (a) implies that -RoS* is also functional. This shows that RoS is the graph of the partial 
function {A, Ro S,C). If R and S are the graphs of total functions, then dom{R) = A and 
dom{S) = B. By (b), we deduce that dom{R o S) = A. By the first part of (c), RoS is the 
graph of the partial function {A, Ro S,C), which is a total function, since dom{R o S) = A. 
□ 

Proposition 12.4.21 shows that it is legitimate to define the composition of functions, pos- 
sibly partial. Thus, we make the following 
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Definition 2.4.3 Given two functions, f: A ^ B and g: B C, possibly partial, the 
composition of f and g, denoted (7 o /, is the function (possibly partial) 

9° f = {A graphif) o graph{g), C). 

The reader must have noticed that the composition of two functions f: A ^ B and 
g: B ^ C is denoted g o whereas the graph oi g o f is denoted graph{f) o graph{g). This 
"reversal" of the order in which function composition and relation composition are written 
is unfortunate and somewhat confusing. 

Once again, we are victim of tradition. The main reason for writing function composition 
as g o f is that traditionally, the result of applying a function / to an argument x is written 
/(x). Then, {go f){x) = g{f{x)), which makes sense. Some people, in particular algebraists, 
write function composition as fog, but then, they write the result of applying a function / 
to an argument x as xf. With this convention, x{f o g) = {xf )g, which also makes sense. 

We prefer to stick to the convention where we write f{x) for the result of applying a 
function / to an argument x and, consequently, we use the notation go f for the composition 
of / with g, even though it is the opposite of the convention for writing the composition of 
relations. 

Given any three relations, R<OAxB,S^BxC and T 'Z C x D, the reader should 
verify that 

{RoS)oT = Ro{SoT). 

We say that composition is associative. Similarly, for any three functions (possibly partial), 
/: A ^ B, g: B —>■ C and h: C ^ D, we have (associativity of function composition) 

{hog)o f = ho{go f). 

2.5 Recursion on N 

The following situation often occurs: We have some set. A, some fixed element, a & A, some 
function, g: A ^ A, and we wish to define a new function, h: N ^ A, so that 

h{0) = a, 
h{n + 1) = g{h{n)) for all G N. 

This way of defining h is called a recursive definition (or a definition by primitive recur- 
sion). I would be surprised if any computer scientist had any trouble with this "definition" 
of h but how can we justify rigorously that such a function exists and is unique? 

Indeed, the existence (and uniqueness) of h requires proof. The proof, although not really 
hard, is surprisingly involved and, in fact quite subtle. For those reasons, we will not give a 
proof of the following theorem but instead the main idea of the proof. The reader will find 
a complete proof in Enderton [2] (Ghapter 4). 
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Theorem 2.5.1 (Recursion Theorem on N) Given any set, A, any fixed element, a & A, 
and any function, g: A —>■ A, there is a unique function, h: N —>■ A, so that 



Proof . The idea is to approximate h. To do this, define a function, /, to be acceptable iff 

1. dom{f) C N and range{f) C A; 

2. If G dom{f), then /(O) = a; 

3. If n + 1 G dom{f), then n G dom{f) and f{n + 1) = g{f{n)). 
Let be the collection of all acceptable functions and set 



All we can say, so far, is that h is a relation. We claim that h is the desired function. For 
this, four things need to be proved: 

1. The relation h is function. 

2. The function h is acceptable. 

3. The function h has domain N. 

4. The function h is unique. 

As expected, we make heavy use of induction in proving (1), (2), (3) and (4). For complete 
details, see Enderton [Hj (Chapter 4). □ 

Theorem 12.5.11 is very important. Indeed, experience shows that it is used almost as 
much as induction! As an example, we show how to define addition on N. Indeed, at the 
moment, we know what the natural numbers are but we don't know what are the arithmetic 
operations such as + or *! (at least, not in our axiomatic treatment; of course, nobody needs 
an axiomatic treatment to know how to add or multiply). 

How do we define m + n, where m, n G N? 

If we try to use Theorem 12.5.11 directly, we seem to have a problem, because addition is 
a function of two arguments, but h and g in the theorem only take one argument. We can 
overcome this problem in two ways: 

(1) We prove a generalization of Theorem 12.5.11 involving functions of several arguments, 
but with recursion only in a single argument. This can be done quite easily but we 
have to be a little careful. 



MO) 

h{n + l) 



g{h{n)) 



a 



for all G N. 
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(2) For any fixed m, we define addm{n) as addm{n) = m + n, that is, we define addition 
of a fixed m to any n. Then, we let m + = addm{n). 

Since solution (2) involves much less work, we follow it. Let 5* denote the successor 
function on N, that is, the function given by 

S{n) = = n + 1. 

Then, using Theorem 12.5.11 with a = m and g = S, we get a function, addm, such that 

addm{0) = m, 
addm{n + 1) = S{addm{n)) = addm{n) + 1 for all n G N. 

Finally, for all m, n G N, we define m + n hj 

m + n = addm{n). 

Now, we have our addition function on N. But this is not the end of the story because we 
don't know yet that the above definition yields a function having the usual properties of 
addition, such as 

m + = m 
m + n = n + m 
[m + n) + p = m + {n + p). 

To prove these properties, of course, we use induction! 

We can also define multiplication. Mimicking what we did for addition, define multm{n) 
by recursion as follows; 

muUm{0) = 0, 
multmin + 1) = multm{n) + m for all n G N. 

Then, we set 

m ■ n = multm{n). 

Note how the recursive definition of multm uses the adddition function, +, previously defined. 
Again, to prove the usual properties of multiplication as well as the distributivity of ■ over 
+, we use induction. Using recursion, we can define many more arithmetic functions. For 
example, the reader should try defining exponentiation, m". 

We still haven't defined the usual ordering on the natural numbers but we will do so in 
the next chapter. Of course, we all know what it is and we will not refrain from using it. 
Still, it is interesting to give such a definition in our axiomatic framework. 
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Given a function, f : A ^ B (possibly partial), with A 7^ 0, suppose there is some function, 
g: B ^ A (possibly partial), called a left inverse of f, such that 

go f ^ idA. 

If such a g exists, we see that / must be total but more is true. Indeed, assume that 
/(a) = f{b). Then, by applying g, we get 

igof)ia)^gifia))^gifib))^igof)ib). 

However, since g o f — id^, we have {g o /)(a) = id^(a) = a and {g o /)(6) = id^(a) = b, so 
we deduce that 

a = b. 

Therefore, we showed that if a function, /, with nonempty domain, has a left inverse, then 
/ is total and has the property that for all a,b E A, f{a) = f{b) implies that a = 6, or 
equivalently a ^ b implies that /(a) 7^ fib). We say that / is injective. As we will see later, 
injectivity is a very desirable property of functions. 

Remark: If A = 0, then / is still considered to be injective. In this case, g is the empty 
partial function (and when B = 0, both / and g are the empty function from to itself). 

Now, suppose there is some function, h: B ^ A (possibly partial), with S 7^ 0, called a 
right inverse of f, but this time, we have 

f oh^ idfi. 

If such an h exists, we see that it must be total but more is true. Indeed, for any b E B, as 
f o h = ids, we have 

f{h{b)) = {foh){b)=idB{b)=b. 

Therefore, we showed that if a function, /, with nonempty codomain has a right inverse, 
h, then h is total and / has the property that for all b E B, there is some a E A, namely, 
a = h{b), so that /(a) = b. In other words, Q'(/) = -B or equivalently, every element in B is 
the image by / of some element of A. We say that / is surjective. Again, surjectivity is a 
very desirable property of functions. 

Remark: If = 0, then / is still considered to be surjective but h is not total unless A — 0, 
in which case / is the empty function from to itself. 

If a function has a left inverse (respectively a right inverse), then it may have more than 
one left inverse (respectively right inverse). 
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If a function (possibly partial), f:A-^B, with A,B ^ ^, happens to have both a left 
inverse, g: B ^ A, and a right inverse, h: B —>■ A, then we know that / and h are total. 
We claim that g = h, so that g is total and moreover g is uniquely determined by /. 

Lemma 2.6.1 Let f : A ^ B be any function and suppose that f has a left inverse, 

g: B ^ A, and a right inverse, h: B A. Then, g = h and moreover, g is unique, which 

means that if g' : B ^ A is any function which is both a left and a right inverse of f , then 

g' = g- 

Proof . Assume that 

g o f = id-A and f o k = ids- 
Then, we have 

g = g o ids = g o {f o h) = {g o f) o h = idA o h = h. 

Therefore, g = h. Now, if g' is any other left inverse of / and h' is any other right inverse of 
/, the above reasoning applied to g and h' shows that g = h' and the same reasoning applied 
to g' and h' shows that g' = h' . Therefore, g' = h' = g = h, that is, g is uniquely determined 
by /. □ 

This leads to the following definition. 

Definition 2.6.2 A function, f: A B, is said to be invertible iff there is a function, 
g: B ^ A, which is both a left inverse and a right inverse, that is, 

5f o / = idA and f og = ids- 

In this case, we know that g is unique and it is denoted f~^. 

From the above discussion, if a function is invertible, then it is both injective and sur- 
jective. This shows that a function generally does not have an inverse. In order to have 
an inverse a function needs to be injective and surjective, but this fails to be true for many 
functions. It turns out that if a function is injective and surjective then it has an inverse. 
We will prove this in the next section. 

The notion of inverse can also be defined for relations, but it is a somewhat weaker notion. 

Definition 2.6.3 Given any relation, R C A x B, the converse or inverse of R is the 
relation, R~^ ^ B x A, defined by 

R-^ = {{b,a) e B X A \ {a, b) E R}. 

In other words, R~^ is obtained by swapping A and B and reversing the orientation of 
the arrows. Figure 12.41 below shows the inverse of the relation of Figure 12.11 
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o as 




Figure 2.4: The inverse of the relation, R, from Figure 12.11 



Now, if R is the graph of a (partial) function, /, beware that R~^ is generally not the 
graph of a function at all, because R~^ may not be functional. For example, the inverse of 
the graph G in Figure 12.21 is not functional, see below: 




Figure 2.5: The inverse, G ^, of the graph of Figure [22] 



The above example shows that one has to be careful not to view a function as a relation 
in order to take its inverse. In general, this process does not produce a function. This only 
works if the function is invertible. 

Given any two relations, R O A x B and S B x C , the reader should prove that 

{RoS)-^ = S-^oR-\ 

(Note the switch in the order of composition on the right hand side.) Similarly, if /: A ^ B 
and g: B ^ C are any two invertible functions, then g o f is invertible and 



{g o f)-' = r' o g-\ 
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2.7 Injections, Surjections, Bijections, Permutations 

We encountered injectivity and surjectivity in Section 12.61 For the record, let us give 

Definition 2.7.1 Given any function, /: A ^ B, we say that / is injective (or one-to-one) 
iff for all a,b & A, if /(a) = f{b), then a = 6, or equivalently, if a 7^ 6, then /(a) 7^ /(&)• We 
say that / is surjective (or onto) iff for every b & B, there is some a E A so that b = f{a), 
or equivalently if Q'(/) = B. The function / is bijective iff it is both injective and surjective. 
When A = B, a. bijection /: A ^ A is called a permutation of A. 



Remarks: 

1. li A = 0, then any function, / : ^ 5 is (trivially) injective. 

2. If B = 0, then / is the empty function from to itself and it is (trivially) surjective. 

3. A function, f : A ^ B, is not injective iff there exist a,b E A with a ^ b and yet 
f{a) = fib). 

4. A function, /: A ^ B, is not surjective iff for some b E B, there is no a G A with 
b = fia). 

5. Since 3/ = {b E B \ (3a G A){b = /(a))}, a function f : A ^ B is always surjective 
onto its image. 

6. The notation f : A "-^ B is often used to indicate that a function, f : A B, is an 
injection. 

7. Observe that if A 7^ 0, a function / is surjective iff f^^{b) 7^ for all b E B. 

8. When A is the finite set A = {1, . . . ,n}, also denoted [n], it is not hard to show that 
there are n\ permutations of [n]. 

The function, /i : Z ^ Z, given by /i(a;) = x + 1 is injective and surjective. However, 
the function, Z ^ Z, given by f2{x) = is neither injective nor surjective (why?). The 
function, f^: Z Z, given by fsix) = 2x is injective but not surjective. The function, 
/4 : Z ^ Z, given by 

k if X = 2k 
k if X = 2fc + 1 



is surjective but not injective. 



Remark: The reader should prove that if A and B are finite sets, A has m elements and B 
has n elements (so, m < n) then the set of injections from AtoB has 



(n — m]\ 



elements. The following Theorem relates the notions of injectivity and surjectivity to the 
existence of left and right inverses. 
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Theorem 2.7.2 Let f : A B be any function and assume A 7^ 0. 

(a) The function f is injective iff it has a left inverse, g (i.e., a function g: B ^ A so that 
go f = idA). 

(b) The function f is surjective iff it has a right inverse, h (i.e., a function h: B —>■ A so 
that f o h = idB). 

(c) The function f is invertible iff it is injective and surjective. 

Proof, (a) We already proved in Section 12.61 that the existence of a left inverse implies 
injectivity. Now, assume / is injective. Then, for every b G range{f), there is a unique 
Of, G A so that /(ofc) = b. Since A 7^ 0, we may pick some a in A. We define g: B Ahj 

n(b) = 1^'' ^^^^ range{f) 
' \a a b e B - range{f) . 

Then, g{f{a)) = a, since /(a) G range{f) and a is the only element of A so that /(a) = /(a)! 
This shows that g o f = id^, as required. 

(b) We already proved in Section that the existence of a right inverse implies surjec- 
tivity. For the converse, assume that / is surjective. As A 7^ and / is a function (i.e., / is 
total), -B 7^ 0. So, for every b E B, the preimage f^^ib) = {a E A \ f{a) = b} is nonempty. 
We make a function, h: B ^ A, as follows: For each b E B, pick some element Of, G f^^{b) 
(which is nonempty) and let h{b) = ai,. By definition of f~^{b), we have f{ah) = b and so, 

f{h{b)) = f{ab) =b, foiallbeB. 

This shows that f o k = ids, as required. 

(c) If / is invertible, we proved in Section [2^6] that / is injective and surjective. Conversely, 
if / is both injective and surjective, by (a), the function / has a left inverse g and by (b) it 
has a right inverse h. However, by Lemma [2. 6. g = h, which shows that / is invertible. □ 

The alert reader may have noticed a "fast turn" in the proof of the converse in (b). 
Indeed, we constructed the function h by choosing, for each b E B, some element in f~^{b). 
How do we justify this procedure from the axioms of set theory? 

Well, we can't! For this, we need another (historically somewhat controversial) axiom, 
the Axiom of Choice. This axiom has many equivalent forms. We state the following form 
which is intuitively quite plausible: 

Axiom of Choice (Graph Version). 

For every relation, R C A x B, there is a function, f: A ^ B, with graph{f) C R and 
dom{f) = dom{R). 
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We see immediately that the Axiom of choice justifies the existence of the function g in 
part (b) of Theorem I2.7.2I 

Remarks: 

1. Let f: A ^ B and g: B ^ Ahe any two functions and assume that 

go f = idA- 

Thus, / is a right inverse of g and g is a left inverse of /. So, by Theorem 12.7.21 (a) and 
(b), we deduce that / is injective and g is surjective. In particular, this shows that any 
left inverse of an injection is a surjection and that any right inverse of a surjection is 
an injection. 

2. Any right inverse, h, of a surjection, f: A B, is called a section of / (which is an 
abbreviation for cross-section). This terminology can be better understood as follows: 
Since / is surjective, the preimage, f^^ip) = {a E A \ f{h)} of any element b & B is 
nonempty. Moreover, f~^{bi) H /~^(62) = whenever bi 7^ 62- Therefore, the pairwise 
disjoint and nonempty subsets, where b E B, partition A. We can think of A 
as a big "blob" consisting of the union of the sets f^^{b) (called fibres) and lying over 
B. The function / maps each fibre, f~^{b) onto the element, b E B. Then, any right 
inverse, h: B —>■ A, of f picks out some element in each fibre, f^^{b), forming a sort 
of horizontal section of A shown as a curve in Figure 12.61 

3. Any left inverse, g, of an injection, f: A B, is called a retraction of /. The 
terminology refiects the fact that intuitively, as / is injective (thus, g is surjective), B 
is bigger than A and since g o f = id a, the function g "squeezes" B onto A in such a 
way that each point b = f{a) in Q=/ is mapped back to its ancestor a E A. So, B is 
"retracted" onto Ahy g. 




Figure 2.6: A section, h, of a surjective function, /. 



Before discussing direct and inverse images, we define the notion of restriction and ex- 
tension of functions. 
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Definition 2.7.3 Given two functions, f : A ^ C and g: B ^ C , with A C B , we say that 
f is the restriction of g to A if graph{f) C graph{g); we write f = g \ A. In this case, we 
also say that g is an extension of f to B. 

2.8 Direct Image and Inverse Image 

A function, f: X ^ Y, induces a function from 2^ to 2^ also denoted / and a function 
from 2^ to 2^ , as shown in the following definition: 

Definition 2.8.1 Given any function, f: X ^ Y, we define the function /: 2"^ ^ 2^ so 
that, for every subset A of X, 

f{A) = {yeY\3xeA,y = f{x)}. 

The subset, f{A), of Y is called the direct image of A under /, for short, the image of A 
under f. We also define the function : 2^ 2^ so that, for every subset B of Y, 

f-\B) = {xeX\3yeB,y = f{x)}. 

The subset, f^^{B), of X is called the inverse image of A under f or the preimage of A 
under f. 

Remarks: 

1. The overloading of notation where / is used both for denoting the original function 
f : X ^ Y and the new function / : 2^ — >■ 2^ may be slightly confusing. If we observe 
that f{{x}) = {f{x)}, for all x E X, we see that the new / is a natural extension 
of the old / to the subsets of X and so, using the same symbol / for both functions 
is quite natural after all. To avoid any confusion, some authors (including Enderton) 
use a different notation for f{A), for example, flA\. We prefer not to introduce more 
notation and we hope that the context will make it clear which / we are dealing with. 

2. The use of the notation /"^ for the function f~^ : 2^ — 2^ may even be more confusing, 
because we know that f~^ is generally not a function from Y to X. However, it is a 
function from 2^ to 2^ . Again, some authors use a different notation for f~^{B), for 
example, /~^|y4]. Again, we will stick to f~^{B). 

3. The set f{A) is sometimes called the push-forward of A along f and f^^{B) is some- 
times called the pullback of B along f. 

4. Observe that f~^{y) = f~^{{y}), where f~^{y) is the preimage defined just after 
Definition 

5. Although this may seem counter- intuitive, the function f~^ has a better behavior than 
/ with respect to union, intersection and complementation. 
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Some useful properties of / : 2 — 5> 2 and / ^ : 2^ 2 are now stated without proof. 
The proofs are easy and left as exercises. 

Proposition 2.8.2 Given any function, /: X ^ Y , the following properties hold: 

(1) For any B C Y , we have 

fif-\B)) C B. 

(2) If f : X ^ Y is surjective, then 

f{f-\B)) = B. 

(3) For any A C X , we have 

ACf~\fiA)). 

(4) If f : X ^ Y is injective, then 

A = f-\f{A)). 

The next proposition deals with the behavior of f: 2^ ^ 2^ and f~^: 2^ 2^ with 
respect to union, intersection and complementation. 

Proposition 2.8.3 Given any function, f: X ^Y, the following properties hold: 
(1) For all A,B <Z X , we have 

f{AUB) = f{A)Uf{B). 

(2) 

f{AnB)Cf{A)nf{B). 
Equality holds if f : X ^ Y is injective. 

(3) 

f{A)-f{B)Cf{A-B). 
Equality holds if f : X Y is injective. 

(4) For all C,D (lY, we have 

f-\CUD) = f-\C)Uf-\D). 

(5) 

f-\cnD) = f-\c)nf-\D). 

(6) 

f-\C-D) = f-\C)-f-\D). 

As we can see from Proposition I2.8.^?l the function f~^ : 2^ — > 2"^ has a better behavior 
than / : 2"^ — 2^ with respect to union, intersection and complementation. 
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2.9 Equinumerosity; The Pigeonhole Principle and the 
Schroder— Bernstein Theorem 

The notion of size of a set is fairly intuitive for finite sets but what does it mean for infinite 
sets? How do we give a precise meaning to the questions: 

(a) Do X and Y have the same size? 

(b) Does X have more elements than Y7 

For finite sets, we can rely on the natural numbers. We count the elements in the two 
sets and compare the resulting numbers. If one of the two sets is finite and the other is 
infinite, it seems fair to say that the infinite set has more elements than the finite one. 

But what is both sets are infinite? 

Remcirk: A critical reader should object that we have not yet defined what a finite set is 
(or what an infinite set is). Indeed, we have not! This can be done in terms of the natural 
numbers, but for the time being, we will rely on intuition. We should also point out that 
when it comes to infinite sets, experience shows that our intuition fails us miserably. So, we 
should be very careful. 

Let us return to the case where we have two infinite sets. For example, consider N and 
the set of even natural numbers, 2N = {0,2,4,6, . . .}. Clearly, the second set is properly 
contained in the first. Does that make N bigger? On the other hand, the function n i— > 2n is 
a bijection between the two sets, which seems to indicate that they have the same number of 
elements. Similarly, the set of squares of natural numbers. Squares = {0, 1, 4, 9, 16, 25, . . .} 
is properly contained in N, yet many natural numbers are missing from Squares. But, the 
map n ^ is a. bijection between N and Squares, which seems to indicate that they have 
the same number of elements. 

A more extreme example is provided by Nx N and N. Intuitively, Nx N is two-dimensional 

and N is one-dimensional, so N seems much smaller than N x N. However, it is possible to 
construct bijections between N x N and N (try to find one!). In fact, such a function, J, has 
the graph partially showed below: 
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The function J corresponds to a certain way of enumerating pairs of integers. Note that 
the value of m + n is constant along each diagonal, and consequently, we have 

J{m,n) = 1 + 2 + ■ ■ ■ + (m + n) + m, 

= ((m + n)(m + n + 1) + 2m)/2, 
= ((m + n)^ + 3m + n)/2. 

For example, J(2, 1) = ((2 + l)^ + 3 • 2 + l)/2 = (9 + 6 + l)/2 = 16/2 = 8. The function 

J(m, n) — -{{m + n)^ + 3m + n) 

is a bijection but that's not so easy to prove! 

Perhaps even more surprising, there are bijections between N and Q. What about between 
M X M and M? Again, the answer is yes, but that's a lot harder to prove. 

These examples suggest that the notion of bijection can be used to define rigorously when 
two sets have the same size. This leads to the concept of equinumerosity. 

Definition 2.9.1 A set A is equinumerous to a set B, written A ^ B,iS there is a bijection 

/: A ^ B. We say that A is dominated by B, written A ^ B, iS there is an injection from 
A to B. Finally, we say that A is strictly dominated by B, written A -< B, iS A ^ B and 
A^f^ B. 

Using the above concepts, we can give a precise definition of finiteness. Firstly, recall 
that for any n e N, we defined [n] as the set [n] = {1, 2, . . . , n}, with [0] = 0. 

Definition 2.9.2 A set. A, is finite if it is equinumerous to a set of the form [n], for some 
n e N. We say that A is countable (or denumerable) iff A is dominated by N. 

Two pretty results due to Cantor (1873) are given in the next Theorem. These are among 
the earliest results of set theory. We assume that the reader is familiar with the fact that 
every number, a; G M, can be expressed in decimal expansion (possibly infinite). For example, 

TT = 3.14159265358979 • • • 

Theorem 2.9.3 (Cantor) (a) The set N is not equinumerous to the set M of real numbers, 
(b) No set, A, is equinumerous to its power set, 2^. 

(a) We use a famous proof method due to Cantor and known as a diagonal argument. 
We will prove that if we assume that there is a bijection, /: N — > M, then there is a real 
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number z not belonging to the image of /, contradicting the surjectivity of /. Now, if / 
exists, we can form a bi-infinite array 

/(O) = ko.doidQ2do3do4^ • • • , 
/(I) = ki.diidi2di3di4 - ■ ■ , 
/(2) = k2.d21d22d23.d2i ■ ■ ■ , 

f{n) = kn.dnldn2 " " " dnn+1 ' ' ' , 



where kn is the integer part of f{n) and the dni are the decimals of f{n), with i > 1. 
The number 

z = 0.(iic?2'i3 ■ ■ ■ dfi-\-i ' ' ' 

is defined as follows: dn+i = 1 if dnn+i 7^ 1, else dn+i = 2 if dnn+i = 1, for every n > 0, The 
definition of z shows that 

dn+i 7^ dnn+1, for all n > 0, 

which implies that z is not in the above array, i.e., z ^ Q f. 

(b) The proof is a variant of Russell's paradox. Assume that there is a bijection 
g: A —>■ 2^] we construct a set i? C A that is not in the image of g, a contradiction. Consider 
the set 

B = {a e A \ a ^ g{a)}. 
Obviously, B (1 A. However, for every a E A, 

a E B iff a ^ g{a), 

which shows that B ^ g{a) for all a E A, i.e., B is not in the image of g. □ 

As there is an obvious injection of M into M, Theorem 12.9.31 shows that N is strictly 
dominated by M. Also, as we have the injection a ^ {a} from A into 2"^, we see that 
every set is strictly dominated by its power set. So, we can form sets as big as we want by 
repeatedly using the power set operation. 

Remarks: 

1. The proof of part (b) of Theorem 12.9.31 only requires (7 to be a surjection. 

2. In fact, R is equinumerous to 2^, but we will not prove this here. 

The following proposition shows an interesting connection between the notion of power set 
and certain sets of functions. To state this proposition, we need the concept of characteristic 
function of a subset. 
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Given any set, X, for any subset, A, of X, define the characteristic function of A, denoted 
Xa, as the function, xa - X ^ {0, 1}, given by 

/ ^ r 1 if X e A 

In other words, xa tests membership in A: For any x G X, Xa{x) = 1 iff a; G A. Observe that 
we obtain a function, 2^^ — >^ {0, 1}"^, from the power set of X to the set of characteristic 
functions from X to {0, 1}, given by 

X{A) = XA. 

We also have the function, S : {0, 1}"^ 2^ , mapping any characteristic function to the set 
that it defines and given by 

S{f) = {xeX I fix) = 1}, 

for every characteristic function, / G {0, 1}"^. 

Proposition 2.9.4 For any set, X, the function 2"^ ^ {0, 1}'''" from the power set of X 
to the set of characteristic functions on X is a bijection whose inverse is S : {0, 1}'''" ^ 2"^. 

Proof . Simply check that x ° '5 = id and S o x = id, which is straightforward. □ 

In view of Proposition I2.9.4t there is a bijection between the power set 2^ and the set of 
functions in {0, 1}"^. If we write 2 = {0, 1}, then we see that the two sets looks the same! 
This is the reason why the notation 2^ is often used for the power set (but others prefer 
V{X)). 

There are many other interesting results about equinumerosity. We only mention four 
more, all very important. 

Theorem 2.9.5 (Pigeonhole Principle) No set of the form [n] is equinumerous to a proper 
subset of itself, where n eN, 

Proof . Although the Pigeonhole Principle seems obvious, the proof is not. In fact, the proof 
requires induction. We advice the reader to skip this proof and come back to it later after 
we have given more examples of proof by induction. 

Suppose we can prove the following Claim: 

Claim. Whenever a function, /: [n] ^ [n], is an injection, then it is a surjection onto [n] 
(and thus, a bijection). 

Observe that the above Claim implies the Pigeonhole Principle. This is proved by con- 
tradiction. So, assume there is a function, f : [n] [n], such that / is injective and 
53/ = ^ C [n] with A ^ [n], i.e., / is a bijection between [n] and A, a proper subset of 
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[n]. Since /: [n] ^ [n] is injective, by the Claim, we deduce that /: [n] ^ [n] is surjective, 
i.e., 3/ = [n], contradicting the fact that = A ^ [n]. 

It remains to prove by induction on n G N that if / : [^] — ^ [n] is an injection, then it is 
a surjection (and thus, a bijection). For n = 0, f must be the empty function, which is a 
bijection. 

Assume that the induction hypothesis holds for any n > and consider any injection, 
/: [n + 1] — s> [n + 1]. Observe that the restriction of / to [n] is injective. 

Case 1. The subset [n] is closed under /, i.e., /(M) C [n]. Then, we know that / \ [n] 
is injective and by the induction hypothesis, /(M) = [n]. Since / is injective, we must have 
f{n + 1) = n + 1. Hence, / is surjective, as claimed. 

Case 2. The subset [n] is not closed under /, i.e., there is some p < n such that 
f{p) = n + 1. We can create a new injection, /, from [n + 1] to itself with the same image 
as / by interchanging two values of / so that [n] closed under /. Define / by 

/(p) = /(n+l) 
f{n + l) = f{p) = n + l 

fi^) = 1 <i <n, i ^ p. 

Then, / is an injection from [n + 1] to itself and [n] is closed under /. By Case 1, / is 
surjective, and as ^3/ = Qf, we conclude that / is also surjective. □ 

Corollary 2.9.6 (Pigeonhole Principle for finite sets) No finite set is equinumerous to a 
proper subset of itself. 

Proof. To say that a set. A, is finite is to say that there is a bijection, g: A —>■ [n], for some 
72 G N. Assume that there is a bijection, /, between A and some proper subset of A. Then, 
consider the function g ° f o g~^, from [n] to itself. The rest of proof consists in showing 
that [n] would be equinumerous to a proper subset of itself, contradicting Theorem 12.9.51 
We leave the details as an exercise. □ 

The pigeonhole principle is often used in the following way: If we have m distinct slots 
and n > m distinct objects (the pigeons), then when we put all n objects into the m slots, 
two objects must end up in the same slot. This fact was apparently first stated explicitly by 
Dirichlet in 1834. As such, it is also known as Dirichlet's box principle. 

Let A be a finite set. Then, by definition, there is a bijection, /: A [n], for some 
n G N. We claim that such an n is unique. Otherwise, there would be another bijection, 
g: A ^ [p], for some p G N with n ^ p. But now, we would have a bijection g o f~^ between 
[n] and [p] with n ^ p. This would imply that there is either an injection from [n] to a 
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proper subset of itself or an injection from [p] to a proper subset of itself jl| contradicting the 
Pigeonhole Principle. 

If A is a finite set, the unique natural number, n G N, such that A ^ [n] is called the 
cardinality of n and we write \A\ = n (or sometimes, card(A) = n). 

Remark: The notion of cardinality also makes sense for infinite sets. What happens is that 
every set is equinumerous to a special kind of set (an initial ordinal) called a cardinal number 
but this topic is beyond the scope of this course. Let us simply mention that the cardinal 
number of N is denoted (say "aleph" 0). 

Corollary 2.9.7 (a) Any set equinumerous to a proper subset of itself is infinite, 
(b) The set N is infinite. 

Proof . Left as an exercise to the reader. □ 

Let us give another application of the pigeonhole principle involving sequences of integers. 
Given a finite sequence, S, of integers, ai, . . . , a„, a subsequence of S is a. sequence, hi, ... , bm, 
obtained by deleting elements from the original sequence and keeping the remaining elements 
in the same order as they originally appeared. More precisely, bi, . . . ,bm is a subsequence 
of Oi, . . . , a„ if there is an injection, g: {1, . . . , m} {1, . . . , n}, such that bi = ag{i) for all 
i G {l,...,m} and i < j implies g{i) < g{j) for all i,j G {l,...,m}. For example, the 
sequence 

19 10 8375264 

contains the subsequence 

9 8 6 4. 

An increasing subsequence is a subsequence whose elements are in strictly increasing order 
and a decreasing subsequence is a subsequence whose elements are in strictly decreasing 
order. For example, 9 8 6 4 is a decreasing subsequence of our original sequence. We now 
prove the following beautiful result due to Erdos and Szekeres: 

Theorem 2.9.8 (Erdos and Szekeres) Letn be any nonzero natural number. Every sequence 
o/n^ + 1 pairwise distinct natural numbers must contain either an increasing subsequence or 
a decreasing subsequence of length n + 1. 

Proof. The proof proceeds by contradiction. So, assume there is a sequence, S, of + 1 
pairwise distinct natural numbers so that all increasing or decreasing subsequences of S 
have length at most n. We assign to every element, s, of the sequence, S, a pair of natural 
numbers, {us,ds), called a label, where Ug, is the length of a longest increasing subsequence 

^Recall that n + 1 = {0, 1, . . . , n} = [n] U {0}. Here in our argument, we are using the fact that for any 
two natural numbers n,p, either n C p or p C n. This fact is indeed true but requires a proof. The proof 
uses induction and some special properties of the natural numbers implied by the definition of a natural 
number as a set that belongs to every inductive set. For details, see Enderton [T3], Chapter 4. 
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of S that starts at s and where dg is the length of a longest decreasing subsequence of S that 
starts at s. 

Since there are no increasing or descreasing subsequences of length n + 1 in S", observe 
that 1 < Us,ds < n for all s E S. Therefore, 

Claim 1: There are at most distinct labels {ug, dg), where s E S. 

We also assert 

Claim 2: If s and t are any two distinct elements of S, then {ug, dg) ^ {ut, dt). 

We may assume that s precedes t m S since otherwise, we interchange s and t in the 
following argument. Since s ^ there are two cases: 

(a) s <t. In this case, we know that there is an increasing subsequence of length Ut starting 
with t. If we insert s in front of this subsequence, we get an increasing subsequence 
oi Ut + 1 elements starting at s. Then, as Ug is the maximal length of all increasing 
subsequences starting with s, we must have Ut + 1 < Ug, i.e., 

Ug > Ut, 

which implies {ug,dg) ^ {ut,dt). 

(b) s > t. This case is similar to case (a), except that we consider a decreasing subsequence 
of length dt starting with t. We conclude that 

dg > dt 

which implies {us,dg) ^ {ut,dt). 
Therefore, in all cases, we proved that s and t have distinct labels. 

Now, by Claim 1, there are only v? distinct labels and S has + 1 elements so, by the 
Pigeonhole Principle, two elements of 5* must have the same label. But, this contradicts 
Claim 2, which says that distinct elements of S have distinct labels. Therefore, S must have 
either an increasing subsequence or a decreasing subsequence of length n + 1, as originally 
claimed. □ 

Remark: Note that this proof is not constructive in the sense that it does not produce the 
desired subsequence; it merely asserts that such a sequence exists. 

Our next theorem is the historically famous Schroder-Bernstein Theorem, sometimes 
called the "Cantor-Bernstein Theorem." Cantor proved the theorem in 1897 but his proof 
used a principle equivalent to the axiom of choice. Schroder announced the theorem in an 
1896 abstract. His proof, published in 1898, had problems and he published a correction 
in 1911. The first fully satisfactory proof was given by Felix Bernstein and was published 
in 1898 in a book by Emile Borel. A shorter proof was given later by Tarski (1955) as a 
consequence of his fixed point theorem. We postpone giving this proof until the section on 
lattices (see Section H^D- 
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Theorem 2.9.9 (Schroder- Bernstein Theorem) Given any two sets, A and B, if there is 
an injection from A to B and an injection from B to A, then there is a bijection between A 
and B. Equivalently, if A ^ B and B ^ A, then A ^ B. 

The Schroder-Bernstein Theorem is quite a remarkable resuh and it is a main tool to 
develop cardinal arithmetic, a subject beyond the scope of this course. 

Our third theorem is perhaps the one that is the more surprising from an intuitive point 
of view. If nothing else, it shows that our intuition about infinity is rather poor. 

Theorem 2.9.10 If A is any infinite set, then A x A is equinumerous to A. 

Proof . The proof is more involved than any of the proofs given so far and it makes use of 
the axiom of choice in the form known as Zorn's Lemma (see Theorem 14.1.30 . For these 
reasons, we omit the proof and instead refer the reader to Enderton [14] (Chapter 6). □ 

In particular. Theorem 12 . 9 . 1 01 implies that MxM is in bijection with R. But, geometrically, 
M X M is a plane and M is a line and, intuitively it is surprising that a plane and a line would 
have "the same number of points." Nevertheless, that's what mathematics tells us! 

Our fourth theorem also plays an important role in the theory of cardinal numbers. 

Theorem 2.9.11 (Cardinal comparability) Given any two sets, A and B, either there is an 
injection from A to B or there is an injection from B to A (that is, either A < B or B < A). 

Proof . The proof requires the axiom of choice in a form known as the Well-Ordering The- 
orem, which is also equivalent to Zorn's lemma. For details, see Enderton [Hj (Chapters 6 
and 7). □ 

Theorem 12.9.101 implies that there is a bijection between the closed line segment 

[0, 1] = {x G R I < a; < 1} 

and the closed unit square 

[0,l]x[0,l] = {(x,i/)GM2|0<a;,i/<l} 

As an interlude, in the next section, we describe a famous space-filling function due to 
Hilbert. Such a function is obtained as the limit of a sequence of curves that can be defined 
recursively. 
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Figure 2.7: A sequence of Hilbert curves ho, hi, /i2 

2.10 An Amazing Surjection: Hilbert 's Space Filling 
Curve 

In the years 1890-1891, Giuseppe Peano and David Hilbert discovered examples of space 
filling functions (also called space filling curves). These are surjective functions from the 
line segment, [0, 1] onto the unit square and thus, their image is the whole unit square! 
Such functions defy intuition since they seem to contradict our intuition about the notion of 
dimension, a line segment is one-dimensional, yet the unit square is two-dimensional. They 
also seem to contradict our intuitive notion of area. Nevertheless, such functions do exist, 
even continuous ones, although to justify their existence rigouroulsy requires some tools 
from mathematical analysis. Similar curves were found by others, among which we mention 
Sierpinski, Moore and Gosper. 

We will describe Hilbert's scheme for constructing such a square-filling curve. We define a 
sequence, {hn), of polygonal lines, hn : [0, 1] — >■ [0, 1] x [0, 1], starting from the simple pattern 
ho (a "square cap" □) shown on the left in Figure 12.71 

The curve /i„+i is obtained by scaling down hn by a factor of |, and connecting the 
four copies of this scaled-down version of /i„ obtained by rotating by 7r/2 (left lower part), 
rotating by — 7r/2 and translating right (right lower part), translating up (left upper part), 
and translating diagonally (right upper part), as illustrated in Figure [221 

It can be shown that the sequence (/i„) converges (uniformly) to a continuous curve 
h: [0, 1] [0, 1] X [0, 1] whose trace is the entire square [0, 1] x [0, 1]. The Hilbert curve h 
is surjective, continuous, and nowhere differentiable. It also has infinite length! The curve 
h^ is shown in Figure 12. 8[ You should try writing a computer program to plot these curves! 
By the way, it can be shown that no continuous square-filling function can be injective. It 
is also possible to define cube-filling curves and even higher-dimensional cube-filling curves! 
(see some of the web page links in the home page for CSE260) 

Before we close this chapter and move on to special kinds of relations, namely, partial 
orders and equivalence relations, we illustrate how the notion of function can be used to 
define strings, multisets and indexed families rigorously. 
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Figure 2.8: The Hilbert curve 

2.11 Strings, Multisets, Indexed Families 

Strings play an important role in computer science and linguistics because they are the basic 
tokens that languages are made of. In fact, formal language theory takes the (somewhat 
crude) view that a language is a set of strings (you will study some formal language theory 
in CSE262). A string is a finite sequence of letters, for example "Jean", "Val", "Mia", 
"math" , "gaga" , "abab" . Usually, we have some alphabet in mind and we form strings using 
letters from this alphabet. Strings are not sets, the order of the letters matters: "abab" 
and "baba" are different strings. What matters is the position of every letter. In the string 
"aba", the leftmost "a" is in position 1, "b" is in position 2 and the rightmost "b" is in 
position 3. All this suggests defining strings as certain kinds of functions whose domains are 
the sets [n] = {1, 2, . . . , n} (with [0] = 0) encountered earlier. Here is the very beginning of 
the theory of formal languages. 

Definition 2.11.1 An alphabet, E, is any finite set. 

We often write E = {ai, . . . , Uk}- The are called the symbols of the alphabet. 

Renicirk: There will be a few occasions where we will allow infinite alphabets but normally 
an alphabet is assumed to be finite. 

Examples: 



S = {a} 
E = {a, b, c} 
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E = {0,1} 

A string is a finite sequence of symbols. Technically, it is convenient to define strings as 
functions. 

Definition 2.11.2 Given an alphabet, S, a string over T, (or simply a string) of length n 
is any function 

u: [n] — > S. 

The integer n is the length of the string, u, and it is denoted by \u\. When n — 0, the special 
string, u: [0] — > E, of length is called the empty string, or null string, and is denoted by e. 

Given a string, u: [n] ^ S, of length n > 1, u{i) is the i-th letter in the string u. For 
simplicity of notation, we denote the string u as 

U = UxU2 ...Un, 

with each e E. 

For example, if E = {a, 6} and [3] — > E is defined such that u{l) — a, u{2) — b, and 
u{3) — a, we write 

u = aba. 

Strings of length 1 are functions m: [1] — > S simply picking some element = in E. 
Thus, we will identify every symbol G S with the corresponding string of length 1. 

The set of all strings over an alphabet E, including the empty string, is denoted as E*. 
Observe that when E = 0, then 

0* = {e}. 

When E 7^ 0, the set E* is countably infinite. Later on, we will see ways of ordering and 
enumerating strings. 

Strings can be juxtaposed, or concatenated. 

Definition 2.11.3 Given an alphabet, S, given two strings, u: [m] — >■ E and v. [n] — >■ E, 
the concatenation, u ■ v, ( also written uv) of u and v is the string 
uv: [m + n] — > E, defined such that 

/ .\ f uii) if 1 < i < m, 

uv{i) ^ < }■ ^ .f , 1 ^ • ^ , 

(^v{t — m) iim+l<i<m + n. 

In particular, ue = eu = u. 

It is immediately verified that 

u{vw) = {uv)w. 

Thus, concatenation is a binary operation on S* which is associative and has e as an identity. 
Note that generally, uv ^ vu, for example for m = a and v = b. 
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Definition 2.11.4 Given an alphabet E, given any two strings u,v G S* we define the 
following notions as follows: 

u is a prefix of v iff there is some y G S* such that 

V = uy. 

u is a suffix of V iff there is some x G S* such that 

V = xu. 

u is a substring of v iff there are some x,y E T,* such that 

V = xuy. 

We say that u is a proper prefix (suffix, substring) o/f iff -u is a prefix (suffix, substring) 
of V and u ^ v. 

For example, ga is a prefix of gallier, the string lier is a suffix of gallier and all is a 
substring of gallier 

Finally, languages are defined as follows. 

Definition 2.11.5 Given an alphabet S, a language over (or simply a language) is any 
subset, L, of S*. 

The next step would be to introduce various formalisms to define languages, such as 
automata or grammars but you'll have to take CSE262 to learn about these things! 

We now consider multisets. We already encountered multisets in Section 11.21 when we 
defined the axioms of propositional logic. As for sets, in a multiset, the order of elements 
does not matter, but as in strings, multiple occurrences of elements matter. For example, 

{a, a, 6, c, c, c} 

is a multiset with two occurrences of a, one occurrence of b and three occurrences of c. This 
suggests defining a multiset as a function with range N, to specify the multiplicity of each 
element. 

Definition 2.11.6 Given any set, S, a multiset, M , over S is any function, M : S" N. A 
finite multiset, M , over S is any function, M : 5 — > N, such that M(a) 7^ only for finitely 
many a G S". If M(a) = > 0, we say that a appears with mutiplicity k in M. 
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For example, if S — {a, b, c}, we may use the notation {a, a, a, b, c, c} for the multiset 
where a has multiplicity 3, b has multiplicity 1, and c has multiplicity 2. 

The empty multiset is the function having the constant value 0. The cardinality \M\ of 
a (finite) multiset is the number 



Note that this is well-defined since M(a) = for all but finitely many a & S. For example 



We can define the union of multisets as follows: If Mi and M2 are two multisets, then 
Ml U M2 is the multiset given by 

(Ml U M2) (a) - Ml (a) + M2 (a) , for all a e S. 

A multiset. Mi, is a submultiset of a multiset, M2, if Mi(a) < M2(a), for all a E S. The 
difference of Mi and M2 is the multiset. Mi — M2, given by 



Intersection of multisets can also be defined but we will leave this as an exercise. 

Let us now discuss indexed families. The Cartesian product construct, Ai x A2 x • • • x An, 
allows us to form finite indexed sequences, (oi, . . . ,a„), but there are situations where we 
need to have infinite indexed sequences. Typically, we want to be able to consider families 
of elements indexed by some index set of our choice, say /. We can do this as follows: 

Definition 2.11.7 Given any, X, and any other set, /, called the index set, the set of I- 
indexed families (or sequences) of elements from X is the set of all functions, A: I ^ X; 
such functions are usually denoted A = {Ai)iQi. When X is a set of sets, each Ai is some set 
in X and we call (Aj)jg7 a. family of sets (indexed by I). 

Observe that if / = [n] = {1, . . . ,n}, then an /-indexed family is just a string over X. 
When / = N, an N-indexed family is called an infinite sequence or often just a sequence. In 
this case, we usually write (a;„) for such a sequence ((a;„)„gN is we want to be more precise). 
Also, note that although the notion of indexed family may seem less general than the notion 
of arbitrary collection of sets, this is an illusion. Indeed, given any collection of sets, X, we 
may choose the set index set / to be X itself, in wich case X appears as the range of the 
identity function, id: X ^ X. 

The point of indexed families is that the operations of union and intersection can be 
generalized in an interesting way. We can also form infinite Cartesian products, which are 
very useful in algebra and geometry. 




\{a, a, a, b, c, c}\ — 6. 
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Given any indexed family of sets, (Aj)jg/, the union of the family {Ai)i^i, denoted IJ^^^ Ai, 
is simply the union of the range of A, that is, 

I^Ai = \^range{A) = {a \ (Bi e I), a e Ai}. 

Observe that when / = 0, the union of the family is the empty set. When / 7^ 0, we say 
that we have a nonempty family (even though some of the Ai may be empty). 

Similarly, if J 7^ 0, then the intersection of the family, (y4j)jg/, denoted f^-g^ Ai, is simply 
the intersection of the range of A, that is, 

Pl^i = ^range{A) = {a \ (Vi E I), aE Ai}. 

i&I 

Unlike the situation for union, when J = 0, the intersection of the family does not exist. It 
would be the set of all sets, which does not exist. 

It is easy to see that the laws for union, intersection and complementation generalize to 
families but we will leave this to the exercises. 

An important construct generalizing the notion of finite Cartesian product is the product 
of families. 

Definition 2.11.8 Given any family of sets, (Aj)^^/, the product of the family (Aj)^^/, de- 
noted Hie/ 

\[A, = {a: I (Vz E I), a{i) E A,}. 

Definition 12.11.81 says that the elements of the product Yliei functions, 
a: I [j-^jAi, such that a{i) E Ai for every i E I. We denote the members of Hie/^* 
by (cij)je/ and we usually call them I-tuples. When I = {1, . . . ,n} = [n], the members of 
riieM functions whose graph consists of the sets of pairs 

{(1, ai), (2, aa), . . . , (n, a„)}, ai E Ai, 1 < i < n, 

and we see that the function 

{(1, ai), (2, 02), ...,{n, an)} ^ (ai, . . . , a„) 

yields a bijection between Iliein] ^« Cartesian product Ai x ■ ■ ■ x An- Thus, if each 

Ai is nonempty, the product nie[n] ^« nonempty. But what if / is infinite? 

If I is infinite, we smell choice functions. That is, an element of Hie/ ^« obtained by 
choosing for every i E I some ai E Ai. Indeed, the axiom of choice is needed to ensure that 
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rije/ 7^ if 7^ for all i E I\ For the record, we state this version (among many!) of 
the axiom of choice: 

Axiom of Choice (Product Version) 

For any family of sets, (Aj)^^/, if J 7^ and Ai ^ for all i e I, then Yl^^j Ai ^ 0. 

Given the product of a family of sets, Hie/^i) foi' each i E I, we have the function 
pTi : Hie/ ^ -^i-i called the ith projection function, defined by 

pri{{ai)ia) = Oj- 



Chapter 3 



Some Counting Problems; Binomial 
Coefficients 

3.1 Counting Permutations and Functions 

In this short section, we consider some simple counting problems. Let us begin with permu- 
tations. Recall that a permutation of a set, A, is any bijection between A and itself. If A is a 
finite set with n elements, we mentioned earlier (without proof) that A has n\ permutations, 
where the factorial function, n i— > n! (ra G N), is given recursively by: 

0! = 1 
{n + l)\ = {n + l)n\. 

The reader should check that the existence of the function, n n\, can be justified using 
the Recursion Theorem (Theorem 12.5.11) . 

Proposition 3.1.1 The number of permutations of a set of n elements is n\. 

Proof . We prove that if A and B are any two finite sets of the same cardinality, n, then the 
number of bijections between A and B is n\. Now, in the special case where B = A, we get 
our theorem. 

The proof is by induction on n. For n = 0, the empty set has one bijection (the empty 
function). So, there are 0! = 1 permutations, as desired. 

Assume inductively that if A and B are any two finite sets of the same cardinality, n, 
then the number of bijections between A and B is n\. li A and B are sets with n + 1 
elements, then pick any element, a & A, and write A = A' U {a}, where A' = A — {a} has 
n elements. Now, any bijection, f : A ^ B, must assign some element of S to a and then 
/ I" A' is a bijection between A' and B' = B — {f (a)} . By the induction hypothesis, there are 
n! bijections between A' and B'. Since there are n + 1 ways of picking /(a) in B, the total 
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number of bijections between A and B is {n + l)n\ = (n + 1)!, establishing the induction 
hypothesis. □ 

Let us also count the number of functions between two finite sets. 

Proposition 3.1.2 If A and B are finite sets with \A\ = m and \B\ = n, then the set of 
function, B^, from A to B has elements. 

Proof. We proceed by induction on m. For m = 0, we have A = 0, and the only function is 
the empty function. In this case, = 1 and the base base holds. 

Assume the induction hypothesis holds for m and assume \ A\ = m + 1. Pick any element, 
a e A, and let A' = A — {a}, a set with m elements. Any function, f: A ^ B, assigns an 
element, /(a) G -B, to a and / |" A' is a function from A' to B. By the induction hypothesis, 
there are functions from A' to B. Since there are n ways of assigning /(a) G -B to a, 
there are n ■ ri^ = ri^^-^ functions from A to B, establishing the induction hypothesis. □ 

As a corollary, we determine the cardinality of a finite power set. 
Corollary 3.1.3 For any finite set, A, if \A\ = n, then |2^| = 2". 

Proof. By proposition 12 .9 .41 there is a bijection between 2^ and the set of functions {0, 1}"^. 
Since |{0, 1}| = 2, we get |2^| = |{0, 1}^| = 2^^, by Proposition [3X1 □ 

Computing the value of the factorial function for a few inputs, say n = 1, 2 . . . , 10, shows 
that it grows very fast. For example. 



It is possible to quantify how fast factorial grows compared to other functions, say n" or e"? 
Remarkably, the answer is yes. A beautiful formula due to James Stirling (1692-1770) tells 
us that 



10! = 3,628,800. 




which means that 



lim 



v^(f) 



n 



1. 



n- 



■oo 



Here, of course. 
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the base of the natural logarithm. It is even possible to estimate the error. It turns out that 





where 



1 



< A„ < 



1 



12n+ 1 



12n 
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a formula due to Jacques Binet (1786-1856). 

Let us introduce some notation used for comparing the rate of growth of functions. We 
begin with the "Big oh" notation. 

Given any two functions, / : N — >• M and gf: N — >• M, we say that / is 0{g) (or f{n) is 
0{g{n)) ) iff there is some N > and a constant c > such that 

\f(n)\<c\g(n)\, for all n>N. 

In other words, for n large enough, |/(n)| is bounded by c|gf(n)|. We sometimes write n >> 
to indicate that n is "large." 

For example A„ is 0{^). By abuse of notation, we often write /(n) = 0[g{n)) even 
though this does not make sense. 

The "Big omega" notation means the following: / is fl{g) (or f{n) is fl{g{n))) iff there 
is some > and a constant c > such that 

|/(n)| > c|^(n)|, for all n>N. 
The reader should check that f{n) is 0{g{n)) iff g{n) is 0.{f{n)). 

We can combine O and fl to get the "Big theta" notation: / is Q{g) (or f{n) is Q^gin)) ) 
iff there is some N > and some constants ci > and C2 > such that 

ci\g{n)\ < \f{n)\ < C2\g{n)\, for all n>N. 

Finally, the "Little oh" notation expresses the fact that a function, /, has much slower 
growth than a function g. We say that / is o{g) (or f{n) is o{g{n)) ) iff 

hm M = 0. 

n-^oo g{n) 

For example, ^/n is o{n). 

3.2 Counting Subsets of Size k\ Binomial Coefficients 

Let us now count the number of subsets of cardinality A; of a set of cardinality n, with 
< A; < n. Denote this number by (^') (say "n choose A;"). Actually, in the proposition 
below, it will be more convenient to assume that /c G Z. 
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Proposition 3.2.1 For all n & N and all k & if (^) denotes the number of subsets of 
cardinality k of a set of cardinality n, then 

1 

if A;^{0,l,...,n} 

("r)-C;:0 

Proof . We proceed by induction on n > 0. Clearly, we may assume that our set is 
[n] = {1, . . . ,n} ([0] = 0). The base case n = is trivial since the empty set is the only 
subset of size 0. When n>l, there are two kinds of subsets of {1, ... , n} having k elements: 
those containing 1, and those not containing 1. Now, there are as many subsets of k elements 
from {1, . . . , n} containing 1 as there are subsets of A; — 1 elements from {2, . . . , n}, namely 
("~^) , and there are as many subsets of k elements from {1, . . . , n} not containing 1 as there 
are subsets of k elements from {2, . . . ,n}, namely ("^^)- Thus, the number of subsets of 
{1, . . . ,n} consisting of k elements is ("^^^) + (^1^); which is equal to (^). □ 

The numbers (^*) are also called binomial coefficients, because they arise in the expansion 
of the binomial expression (a + 6)", as we will see shortly. The binomial coefficients can be 
computed inductively using the formula 

{n\ fn — 1\ fn — 1\ 

(sometimes known as Pascal's recurrence formula) by forming what is usually called Pascal's 
triangle, which is based on the recurrence for (^): 

^ (o) (i) (2) (3) C) (5) (e) (r) ■■■ 

1 

11 1 

2 12 1 

3 13 3 1 

4 14 6 4 

5 1 5 10 10 

6 1 6 15 20 

7 1 7 21 35 




1 

5 1 

15 6 1 

35 21 7 1 



We can also give the following explicit formula for in terms of the factorial function: 
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Proposition 3.2.2 For all n,k eN, with < k < n, we have 



kj k\{n-k)\ 



Proof . Left as an exercise to the reader (use induction on n and Pascal's recurrence formula). 
□ 

Then, it is very easy to see that 

k ) \n — k 



Reiricirk: The binomial coefficients were already known in the twelfth century by the Indian 
Scholar Bhaskra. Pascal's triangle was taught back in 1265 by the Persian philosopher, Nasir- 
Ad-Din. 

We now prove the "binomial formula" (also called "binomial theorem"). 



Proposition 3.2.3 (Binomial Formula) For any two reals a, 6 G M (or more generally, any 
two commuting variables a, b, i.e. satisfying ab = ba), we have the formula: 

(a + by = a" + {^^ a^-'b + (^^ + ' " " + (^) + " ' " + ! i) + ^" 



The above can be written concisely as 



fc=o ^ ^ 



Proof . We proceed by induction on n. For n = 0, we have (a + 6)° = 1 and the sum on the 
righthand side is also 1, since (q) — 1. 

Assume inductively that the formula holds for n. Since 



{a + bY+^ = {a + bY{a + b), 
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using the induction hypothesis, we get 

(a + 6)"+i = (a + 6)"(a + 6) 




,n+l 



where we used Proposition 13.2.11 to go from the next to the last hne to the last line. This 
establishes the induction step and thus, proves the binomial formula. □ 

We also stated earlier that the number of injections between a set with m elements and 
a set with n elements, where m < n, is given by and we now prove it. 

Proposition 3.2.4 The number of injections between a set, A, with m elements and a set, 
B, with n elements, where m < n, is given by ^^"^^| = n{n — 1) ■ ■ ■ {n — m + 1). 

Proof . We proceed by induction on m < n. If m = 0, then A = and there is only one 
injection, namely the empty function from to B. Since j^^^zoy ~ ^ ~ base case holds. 

Assume the induction hypothesis holds for m and consider a set. A, with m + 1 elements, 
where m + 1 < n. Pick any element a & A and let A' = A — {a}, a set with m elements. Any 
injection, f:A-^B, assigns some element, /(a) G -B, to a and then / f A' is an injection 
from A' to B' = B — {f{a)}, a. set with n — 1 elements. By the induction hypothesis, there 
are 

(n- 1)! 
{n — 1 — m)\ 

injections from A' to B'. Since there are n ways of picking /(a) in B, the number of injections 
from y4 to -B is 

(n-1)! n\ 

Tl = 

{n — 1 — m)\ (n — (m + 1))!' 
establishing the induction hypothesis. □ 
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Counting the number of surjections between a set with n elements and a set with p 
elements, where n > p, is harder. We state the following formula without proof, leaving the 
proof as an interesting exercise. 

Proposition 3.2.5 The number of surjections, Snp, between a set, A, with n elements and 
a set, B, with p elements, where n> p, is given by 

Sup = - {P - IT + (2) (p - 2)" + • • • + (-1)^-^ ! . 

Reiricirks: 

1. It can be shown that Snp satisfies the following peculiar version of Pascal's identity: 

Snp — P{Sn-lp + 'S'n-lp-l)- 

2. The numbers, S'np, arc intimately related to the so-called Strirling numbers of the 
second kind, denoted |^}, S{n,p), or Sn \ which count the number of partitions of a 
set of n elements into p nonempty pairwise disjoint blocks. In fact. 




The binomial coefficients can be generalized as follows. For all n, m, /ci, . . . , A;^ e N, with 
ki + ■ • ■ + km — n and m > 2, we have the multinomial coefficient, 

n 

which counts the number of ways of splitting a set of n elements into m disjoint subsets, the 
ith. subset having ki elements. Note that when m = 2, the number of ways splitting a set of 
n elements into two disjoint subsets where one of the two subsets has ki elements and the 
other subset has k2 = n — ki elements is precisely the number of subsets of size ki of a set 
of n elements, that is 

' n \ /n 
M ^2/ \ki, 

Proposition 3.2.6 For all n,m,ki, . . . ,km £ with ki + ■ • ■ + km. — n and m >2, we have 

n \ n\ 



ki ■ ■ ■ kjnj ki \ ■ ■ ■ km}. 
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Proof. There are (^J ways of forming a subset of ki elements from the set of n elements; there 

are ways of forming a subset of ^2 elements from the remaining n — ki elements; there 

are ways of forming a subset of elements from the remaining n — ki — k2 elements 

and so on; finally, there are ("^"'^^fc ~'^"'"^) ways of forming a subset of /cm-i elements from 
the remaining n — /ei — • • ■ — km-2 elements and there remains a set of n — A;i — ■ • ■ — km-i = km 
elements. This shows that 

n fn~ki- km-2 

ki--- km) \kij \ k2 J V ^m-i 

But then, using the fact that km = n — ki — ■ ■ ■ — km-i, we get 

n \ n\ {n — ki)\ in — ki — • ■ ■ — km~2) 



ki---kmj ki\{n - ki)] k2\{n - ki - k2y. km-i\{n - ki km-i)\ 

n\ 



k,^ ■■ - k ' ' 

as claimed. 



As in the binomial case, it is convenient to set 



n 







if fcj < or ki > n, for any i, with 1 < i < m. Then, Proposition 13.2.11 is generalized as 
follows: 

Proposition 3.2.7 For all n, m, ki, . . . , km ^ N, with ki + ■ ■ ■ + km = n, n > 1 and m >2, 

we have 

n \ / n — 1 

M--- ^m) ^\ki---{ki-l)---kr, 

Proof. Note that we have k^ — 1 = —1 when ki = 0. If we observe that 

kJ )=n( ""'^ 

\ki ■ ■ ■ km J \ki ■ ■ ■ {ki 1) ■ ■ ■ A;^ 

even if fcj = 0, then we have 

y^/ n-1 \ - ^ ( ^ 

^ \ ki- ■ ■ {ki- 1) ■ ■ -kmj \n nj\ki---k„ 



i=l 



n 



since ki + ■ ■ ■ + km = n. □ 
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Remark: Proposition 13.2.71 shows that Pascal's triangle generalizes to "higher dimensions", 
that is, to m > 3. Indeed, it is possible to give a geometric interpretation of Proposition 13 . 2 . 71 
in which the multinomial coefficients corresponding to those ki, . . . ,km with ki + - ■ ■ + km = n 
lie on the hyperplane of equation xi + ■ ■ ■ + Xm = n in M™, and all the multinomial coefficients 
for which n < N, for any fixed A^, lie in a generalized tetrahedron called a simplex. When 
m = 3, the multinomial coefficients for which n < lie in a tetrahedron whose faces are the 
planes of equations, x = 0; y = 0; z = 0; and x + y + z = N. 

We have also the following generalization of Proposition 13.2.31 



Proposition 3.2.8 (Multinomial Formula) For all n,m E N with m > 2, for all pairwise 
commuting variables ai, . . . , a^, we have 



Proof. We proceed by induction on n and use Proposition 13.2.71 The case n = is trivially 
true. 

Assume the induction hypothesis holds for n > 0, then we have 
(ai + ■ • ■ + a-mT^^ = (ai + • • ■ + a„)"(ai + • • ■ + a^) 




n 




fciH \-km=n 





fciH hfcm=n 




ki- ■ ■ {ki - 1) ■ ■ - k, 



n 



) 



a 



1 




We seem to hit a snag, namely, that k^ > 1, but recall that 
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so we have 



[ai + ■ ■ ■ + ttr, 



\n+l 



i=l ki,...,km>0,ki>] 
fciH \-km=n+l 



i=l ki,...,km>0, 
fciH \-km=n+l 



n 



/Cl • • • (/c^ 1) • • • kjnj 



ki,...,km>0, \i=l ^ ^ ' 



■ ■ ■ ■ ■ ■ 



fciH |-fcm=n+l 



fcl,...,fcm>0, 

fciH |-A:m=n+1 



^ ~^ \ fci ki k„ 

U k-...k '"^ ■ ■ - "i" ■ ■ - "m. 

fbi ft, J ft.^ 



where we used Proposition 13.2.71 to justify the last equation. Therefore, the induction step 
is proved and so is our proposition. □ 

How many terms occur on the right-hand side of the multinomial formula? After a 
moment of reflexion, we see that this is the number of finite multisets of size n whose 
elements are drawn from a set of m elements, which is also equal to the number of m-tuples, 
ki, . . . , km, with ki E N and 

k\ -\- ' ' ' -\- kfyi = Ti. 

The following proposition is left an exercise: 



Proposition 3.2.9 The number of finite multisets of size n>0 whose elements come from 
a set of size m > 1 is 

+ n — V 



n 



3.3 The Inclusion-Exclusion Principle 

We close this chapter with the proof of a poweful formula for determining the cardinality 
of the union of a finite number of (finite) sets in terms of the cardinalities of the various 
intersections of these sets. This identity variously attributed Nicholas Bernoulli, de Moivre, 
Sylvester and Poincare has many applications to counting problems and to probability theory. 
We begin with the "baby case" of two finite sets. 



Proposition 3.3.1 Given any two finite sets, A, and B, we have 



AVJB\ = \A\ + 151 - \AnB 
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Proof. This formula is intuitively obvious because if some element, a G AU B, belongs to 
both A and B then it is counted twice in |y4| + and so we need to subtract its contribution 
to An B. Nevertherless, it is worth giving a rigorous proof by induction on n = \AU B\. 

If n = 0, then A = i? = and the formula is clear: = — 0. 

For the induction step, assume that AU B has n + 1 elements and pick any element, 
a E AU B. There are three cases: 

1. a G A and a ^ B. Since a ^ B, {A — {a}) Ci B = A H B. By the induction hypothesis, 

\{A-{a})UB\ = \A-{a}\ + \B\ + \AnB\. 
Then, adding a to A — {a} adds 1 to \{A — {a}) U B\ and to \A — {a}\, so we get 

\AUB\ = \A\ + \B\ - \AnB\, 

proving the induction step. 

2. a ^ A and a E B. This case is analogous to the previous one except that the roles of 
A and B are swapped. 

3. a G A and a E B, i.e., a E Ad B. In this case, by the induction hypothesis, we have 

\{A - {a}) U{B- {a})\ = \A - {a}\ + \B - {a}\ + \{A - {a}) n {B - {a})\. (*) 

Adding a to A — {a} and B — {a} adds 1 to \{A — {a}) U {B — {a})\; it also adds 
1 to both \A - {a}\ and \B - {a}\ and adds 1 to \{A - {a}) f] {B - {a})\. So, the 
contribution of a to the righthand side of (*) is 2 — 1 = 1 and we get 

\AUB\ = \A\ + \B\ - \AnB\, 

proving the induction step. □ 

We would like to generalize the formula of Proposition 13.3.11 to any finite collection of 
finite sets, Ai, . . . , An. A moment of reflexion shows that when n = 3, we have 

\Au Buc\ = \A\ + \B\ + \c\ -\AnB\-\Anc\-\Bnc\ + \AnBr]C\. 

One of the obstacles in generalizing the above formula to n sets is purely notational: We 
need a way of denoting arbitrary intersections of sets belonging to a family of sets indexed 
by {1, . . . , n}. We can do this by using indices ranging over subsets of {1, . . . , n}, as opposed 
to indices ranging over integers. So, for example, for any nonempty subset, / C {1, . . . 
the expression f^.^^ A^ denotes the intersection of all the subsets whose index, i, belongs to 
/. 
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Theorem 3.3.2 (Inclusion-Exclusion Principle) For any finite sequence, Ai, . . . , A^, of 

n > 2 subsets of a finite set, X , we have 



fe=i 



/C{l,...,n} 



Proof . We proceed by induction on n > 2. The base case, n = 2, is exactly Proposition 
I3.3.1I Let us now consider the induction step. We can write 

n+l / n \ 

\jAk=l[jAA U{A„+i} 
fe=i \fc=i / 

and so, by Proposition 13.3. we have 

n+l 



k=l 



\jAk]u {An+l} 



Kk=l 



k=l 



+ \An. 



+1| 



[jAk] n{A„+i} 



We can apply the induction hypothesis to the first term and we get 



k=l 



JC{l,...,n} 



Using distributivity of intersection over union, we have 

' n \ n 

U n {A„,+i} = [j{AknA 

\k=l J k=l 

Again, we can apply the induction hypothesis and obtain 



[jiAkHAn- 



k=l 



JC{l,...,n} 



JC{l,...,n} 
JC{l,...,n} 



f|(A,nA„+i; 



n 

j6JU{ri+l} 



(|JU{n+l}|-l) 



n -4. 

ieJu{ri+i} 
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Putting all this together, we get 

n+l 



k=l 



JC{l,...,n} 



\An+l\+ Yl (-l)(l^^^"+^^|-^) 



JC{l,...,n} 



n A 



jeJU{n+l} 



JC{l,...,n+l} 



/C{l,...,n+1} 



JC{l,...,n+l} 
n+leJ 



establishing the induction hypothesis and finishing the proof. □ 

As an application of the Inclusion- Exclusion Principle, let us prove the formula for count- 
ing the number of surjections from {1, . . . , ra} to {1, . . . ,p}, with p < n, given in Proposition 

Recall that the total number of functions from {1, . . . , n} to {1, . . . ,p} is p". The trick is 
to count the number of functions that are not surjective. Any such function has the property 
that its image misses one element from {1, . . . So, if we let 

A, = {/:{l,...,n}^{l,...,p}|z^53(/)}, 

we need to count \Ai U ■ ■ ■ U Ap\. But, we can easily do this using the Inclusion- Exclusion 
Principle. Indeed, for any nonempty subset, J, of {1, . . . ,p}, with |/| = k, the functions in 
Ai are exactly the functions whose range misses /. But, these are exactly the functions 
from {1, . . . , ra} to {1, . . . ,p} — / and there are (p — k)"' such functions. Thus, 



{p - ky. 



As there are (^) subsets, / C {1, . . . with |J| = /c, the contribution of all k-io\d intersec- 
tions to the Inclusion-Exclusion Principle is 



{p - kf. 



Note that Aifl- ■ -flAp = 0, since functions have a nonempty image. Therefore, the Inclusion- 
Exclusion Principle yields 



^iu---uA,i = E(-ir^(^)(p-fcy 
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and so, the number of surjections, Snp, is 

p-i 



Snp = p''-\A,U---UA,\=p--J2i-'^)'~'(^\p'ky 

k=i ^ ^ 

= E(-i)'(l)(p-^r 

fc=0 ^ ^ 



= P"-g(P-ir + ^2j(P-2r + ... + (-i, 

which is indeed the formula of Proposition 13.2. 5[ 

Another amusing apphcation of the Inclusion-Exclusion Principle is the formula giving 
the number, of permutations of {1, . . . ,n} that leave no element fixed (i.e., f{i) ^ i, for 
all i E {I, . . . ,n}). Such permutations are often called derangements. We get 

, A 1 1 (-1)' ^-11 



1! 2! k\ n\ 

n!-(^](n-l)!+('^)(n-2)! + --- + (-ir. 



Remark: We know (using the series expansion for in which we set a; = — 1) that 

1 1 1 (-1)^ 

Consequently, the factor of n! in the above formula for p„ is the sum of the first n + 1 terms 
of - and so, 

e ' 

Pn 1 

hm — = -. 

n— »oo nl e 

It turns out that the series for - converges very rapidly, so Pn ~ -n\. The ratio Pn/n\ has 
an interesting interpretation in terms of probabilities. Assume n persons go to a restaurant 
(or to the theatre, etc.) and that they all check their coats. Unfortunately, the deck loses 
all the coat tags. Then, p„/n! is the probability that nobody will get her or his own coat 
back! As we just explained, this probability is roughly ^ ^ ^, a. surprisingly large number. 

The Inclusion-Exclusion Principle can be easily generalized in a useful way as follows: 
Given a finite set, X, let m be any given function, m: X ^ M_|_, and for any nonempty 
subset, A C X, set 

m{A) = m(a), 

aeA 

with the convention that m(0) = (Recall that M_|_ = {x G M | x > 0}). For any x E X, 
the number m{x) is called the weight (or measure) of x and the quantity m{A) is often 
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called the measure of the set A. For example, if m{x) = 1 for all x & A, then m{A) = \A\, 
the cardinality of A, which is the special case that we have been considering. For any two 
subsets, A,B C X, it is obvious that 



m^AUB) = m{A) + m{B) 

m{X - A) = m{X) - m{A) 

m{A\JB) = m(A n B) 

m{A n B) = m(AUB), 

where A = X — A. Then, we have the following version of Theorem 13.3.21 



Theorem 3.3.3 (Inclusion-Exclusion Principle, Version 2) Given any measure function, 
m: X ^ M.,., for any finite sequence, Ai, . . . , A^, of n > 2 subsets of a finite set, X , we 
have 

\fc=l / IC{l,...,n} J 



Proof . The proof is obtained from the proof of Theorem 13.3.21 by changing everywhere any 
expression of the form to m{B). □ 

A useful corollary of Theorem 13.3.31 often known as Sylvester's formula is: 



Theorem 3.3.4 (Sylvester's Formula) Given any measure, m: X — > M+, for any finite 
sequence, Ai, . . . , An, of n > 2 subsets of a finite set, X , the measure of the set of elements 
of X that do not belong to any of the sets A^ is given by 



m 



/C{l,...,n} 



Proof. Observe that 



n n 

f]Ak = X-[jA,. 

k=l k=l 
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Consequently, using Theorem I3.3.3[ we get 



n \ / n 

m\f]Ak] = mlx-[jAk 

\k=l / \ k=l 

n 



m{X) -mi [jAk 

\k=l 

7C{l,...,n} Vie/ / 

m{X)+ (-l)'""^(n^O' 

7C|l,...,n| Vie/ / 



:{i,...,n} 

establishing Sylvester's formula. □ 

Note that if we use the convention that when the index set, J, is empty then 

r\A=x, 

then the term m{X) can be included in the above sum by removing the condition that 
/ 7^ 0. Sometimes, it is also convenient to regroup terms involving subsets, J, having the 
same cardinality and another way to state Sylvester's formula is as follows: 



n 



m 



Pi Afc = ^(-1)'' Yl mi^AA. (Sylvester's Formula) 

^A:=l / fc=0 /C{l,...,n} Vie/ / 

|/|=fc 

Finally, Sylvester's formula can be generalized to a formula usually known as the "Sieve 
Formula" : 

Theorem 3.3.5 (Sieve Formula) Given any measure, m: X — > M+, for any finite sequence, 
Ai, . . . , An, of n> 2 subsets of a finite set, X , the measure of the set of elements of X that 
belong to exactly p of the sets Ai (0 < p < n) is given by 



k=p ^-^^ ICU....,n} \iel 



A,, 



{l,...,n} 
|/|=fc 



Proof . For any subset, / C {1, . . . , n}, apply Sylvester's formula to X = f]^^j Ai and to the 
subsets Aj n f]-^j Ai. We get 



m 



n^.n[]A)i= y: (-i)'^'-'^'-fn^.) 

,*e/ j0 J jc{i,. ..,«,} VjeJ / 

ICJ 
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Hence, 



/C{l,...,n} HI I 

\I\=p 

IC{l,...,n} JC{l,...,n} \jeJ / 

\I\=p ICJ 

JC{l,...,n} ICJ \jeJ / 

\J\>P \1\=P 

k=p ^-^^ JC{l,...,n} \jeJ / 

\J\=k 

establishing the Sieve formula. □ 

Observe that Sylvester's Formula is the special case of the Sieve Formula for which p = 0. 
The Inclusion-Exclusion Principle (and its relatives) plays an important role in combinatorics 
and probablity theory as the reader will verify by consulting any text on combinatorics. A 
classical reference on combinatorics is Berge [2] ; a more recent is Cameron [8] ; a more recent 
and more advanced is Stanley [39]. Another fascinating (but deceptively tough) reference 
covering discrete mathematics and including a lot of combinatorics is Graham, Knuth and 
Patashnik [21]. 

We are now ready to study special kinds of relations: Partial orders and equivalence 
relations. 



114 CHAPTER 3. SOME COUNTING PROBLEMS; BINOMIAL COEFFICIENTS 



Chapter 4 



Partial Orders, Complete Induction 
and Equivalence Relations 

4.1 Partial Orders 

There are two main kinds of relations that play a very important role in mathematics and 
computer science: 

1. Partial orders 

2. Equivalence relations. 

In this section and the next few ones, we define partial orders and investigate some of 
their properties. As we will sec, the ability to use induction is intimately related to a very 
special property of partial orders known as wcll-foundcdncss. 

Intuitively, the notion of order among elements of a set, X, captures the fact some 
elements are bigger than others, perhaps more important, or perhaps that they carry more 
information. For example, we are all familiar with the natural ordering, <, of the integers 

•••,-3<-2<-l<0<l<2<3<---, 

the ordering of the rationals (where ^ < iff > 0, i.e., p2?i — P1Q2 > if qiq2 > 

else p2qi — Piq2 < if qiq2 < 0), and the ordering of the real numbers. In all of the above 
ordcrings, note that for any two number a and b, either a < b or b < a. We say that such 
orderings are total ordcrings. 

A natural example of an ordering which is not total is provided by the subset ordering. 
Given a set, X, we can order the subsets of X by the subset relation: A C B, where A, B 
are any subsets of X. For example, ii X — {a, b, c}, we have {a} C {a, b}. However, note 
that neither {a} is a subset of {b, c} nor {b, c} is a subset of {a}. We say that {a} and {b, c} 
are incomparable. Now, not all relations are partial orders, so which properties characterize 
partial orders? Our next definition gives us the answer. 
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Definition 4.1.1 A binary relation, <, on a set, X, is a partial order (or partial ordering) 
iff it is reflexive, transitive and antisymmetric, that is: 

(1) (Reflexivity): a < a, for all a E X; 

(2) ( Transitivity): li a < b and b < c, then a < c, for all a,b,c E X. 

(3) (antisymmetry): If a < b and b < a, then a = b, for all a,b E X. 

A partial order is a tota/ order (ordering) (or linear order (ordering)) iff for all a, 6 G X, 
either a < b or b < a. When neither a < b nor 6 < a, we say that a and b are incomparable. 
A subset, C C X, is a chain iff < induces a total order on C (so, for all a,b E C, either 
a <b oi b < a). The strict order (ordering), <, associated with < is the relation defined 
by: a < 6 iff a < 6 and a ^b. If < is a partial order on X, we say that the pair {X, <) is a 
partially ordered set or for short, a poset. 

Remark: Observe that if < is the strict order associated with a partial order, <, then < is 
transitive and anti-reflexive, which means that 

(4) a ^ a, for all a E X. 

Conversely, let < be a relation on X and assume that < is transitive and anti-reflexive. 
Then, we can define the relation < so that a < b iS a = b or a < b. It is easy to check that 
< is a partial order and that the strict order associated with < is our original relation, <. 

Given a poset, {X, <), by abuse of notation, we often refer to {X, <) as the poset X, the 
partial order < being implicit. If confusion may arise, for example when we are dealing with 
several posets, we denote the partial order on X by <x- 

Here are a few examples of partial orders. 

1. The subset ordering. We leave it to the reader to check that the subset relation, 
C, on a set, X, is indeed a partial order. For example, ii A O B and BOA, where 
A, B ^ X , then A = B, since these assumptions are exactly those needed by the 
extensionality axiom. 

2. The natural order on N. Although we all know what is the ordering of the natural 
numbers, we should realize that if we stick to our axiomatic presentation where we 
defined the natural numbers as sets that belong to every inductive set (see Definition 
11.8.31) . then we haven't yet defined this ordering. However, this is easy to do since the 
natural numbers are sets. For any m,n eN, define m < n as m = n or m E n\ Then, 
it is not hard check that this relation is a total order (Actually, some of the details are 
a bit tedious and require induction, see Enderton [H], Chapter 4). 
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3. Orderings on strings. Let S = {«!, . . . ,a„} be an alphabet. The prefix, suffix and 
substring relations defined in Section [2.11l are easily seen to be partial orders. However, 
these orderings are not total. It is sometimes desirable to have a total order on strings 
and, fortunately, the lexicographic order (also called dictionnary order) achieves this 
goal. In order to define the lexicographic order we assume that the symbols in S are 
totally ordered, ai < 02 < ■ ■ ■ < cin- Then, given any two strings, m, f G S*, we set 

if f = uy, for some ?/ G S*, or 
if u = xttiy, V = XQjZ, 
and Qi < aj, for some x,y, z E T,* . 

In other words, either m is a prefix of v or else u and v share a common prefix, x, and 
then there is a differring symbol, in u and aj in v, with < aj. It is fairly tedious 
to prove that the lexicographic order is a partial order. Moreover, the lexicographic 
order is a total order. 

4. The divisibility order on N. Let us begin by defining divisibility in Z. Given any 
two integers, a, 6 G Z, with 6 7^ 0, we say that b divides a {a is a multiple of b) iff 
a = bq for some g G Z. Such a g is called the quotient of a and b. Most number theory 
books use the notation 6 | a to express that b divides a. We leave the veriffcation 
that the divisibility relation is reffexive and transitive as an easy exercise. What about 
antisymmetry? So, assume that b \ a and a \ b (thus, a, 6 7^ 0). This means that there 
exist gi, g2 € ^ so that 

a = bqi and b = aq2. 
From the above, we deduce that b = bqiq2, that is 

- 9192) = 0. 

As 6 7^ 0, we conclude that 

qiq2 = 1- 

Now, let us restrict ourselves to N_|_ = N — {0}, so that a,b > 1. It follows that 
92 £ N and in this case, qiq2 = 1 is only possible iff gi = ^2 = 1- Therefore, a = b 
and the divisibility relation is indeed a partial order on N+. Why is divisibility not a 
partial order on Z — {0}? 

Given a poset, (X <), if X is finite, then there is a convenient way to describe the partial 
order < on X using a graph. In preparation for that, we need a few preliminary notions. 

Consider an arbitrary poset, (X <) (not necessarily finite). Given any element, a G X, 
the following situations are of interest: 

1. For no 6 G X do we have b < a. We say that a is a minimal element (of X). 

2. There is some 6 G X so that b < a and there is no c G X so that b < c < a. We say 
that b is an immediate predecessor of a. 



u < V 
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3. For no 6 e X do we have a < b. We say that a is a maximal element (of X). 

4. There is some b E X so that a < b and there is no c € X so that a < c < b. We say 
that b is an immediate successor of a. 

Note that an element may have more than one immediate predecessor (or more than one 
immediate successor). 

If X is a finite set, then it is easy to see that every element that is not minimal has 
an immediate predecessor and any element that is not maximal has an immediate successor 
(why?). But if X is infinite, for example, X = Q, this may not be the case. Indeed, given 
any two distinct rational numbers, a, 6 G Q, we have 

a + b ^ 

a < < b. 

2 

Let us now use our notion of immediate predecessor to draw a diagram representing a 
finite poset, {X, <). The trick is to draw a picture consisting of nodes and oriented edges, 
where the nodes are all the elements of X and where we draw an oriented edge from a to 6 
iff a is an immediate predecessor of b. Such a diagram is called a Hasse diagram for (X, <). 
Observe that if a < c < 6, then the diagram does not have edges corresponding to the 
relations a < c and c < b. However, such information can be recovered from the diagram 
by following paths consisting of one or several consecutive edges (we are a bit informal here, 
but we will define directed graphs and paths more rigorously later) . Similarly, the self- loops 
corresponding to the the reflexive relations a < a are omitted. A Hasse diagram is an 
economical representation of a finite poset and it contains the same amount of information 
as the partial order, <. 

Here is the diagram associated with the partial order on the power set of the two element 
set, {a, b}: 



{a,b} 








Figure 4.1: The partial order of the power set 2^"'''^ 
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Here is the diagram associated with the partial order on the power set of the three element 
set, {a, b, c}: 



{a, b, c} 






Figure 4.2: The partial order of the power set 2^"'*'''^^ 

Note that is a minimal clement of the above poset (in fact, the smallest element) and 
{a,b,c} is a maximal element (in fact, the greatest element). In the above example, there 
is a unique minimal (resp. maximal) element. A less trivial example with multiple minimal 
and maximal elements is obtained by deleting and {a, b, c}: 




Figure 4.3: Minimal and maximal elements in a poset 

Given a poset, (X, <), observe that if there is some element m G X so that m < x for 
all X & X, then m is unique. Indeed, if m' is another element so that m' < x for all x & X, 
then if we set x — m' in the first case, we get m < m' and if we set a; = m in the second 
case, we get m' < m, from which we deduce that m — m', as claimed. Such an element, m, 
is called the smallest or the least element of X. Similarly, an element, b & X, so that x < b 
for all X & X is unique and is called the greatest element of X. 

We summarize some of our previous definitions and introduce a few more useful concepts 

in 
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Definition 4.1.2 Let (X, <) be a poset and let A C X be any subset of X. An element, 
b E X, is a lower bound of A iS b < a for all a E A. An element, m G X, is an upper bound 
of A iS a < m for all a E A. An element, 6 G X, is the least element of A iS b E A and 
b < a for all a E A. An element, m G X, is the greatest element of A iS m E A and a < m 
for all a G A. An element, 6 G A, is minimal in A iS a < b for no a G A, or equivalently, if 
for all a G A, a < 6 implies that a = b. An element, m G A, is maximal in A iS. m < a for 
no a G v4, or equivalently, if for all a G A, m < a implies that a = m. An element, 6 G X, 
is the greatest lower bound of A iff the set of lower bounds of A is nonempty and if b is the 
greatest element of this set. An element, m G X, is the least upper bound of A iff the set of 
upper bounds of A is nonempty and if m is the least element of this set. 

Remarks: 

1. If 6 is a lower bound of A (or m is an upper bound of A), then b (or m) may not belong 
to A. 

2. The least element of A is a lower bound of A that also belongs to A and the greatest 
element of A is an upper bound of A that also belongs to A. When A = X, the least 
element is often denoted ±, sometimes 0, and the greatest element is often denoted T, 
sometimes 1. 

3. Minimal or maximal elements of A belong to A but they are not necessarily unique. 

4. The greatest lower bound (or the least upper bound) of A may not belong to A. We 
use the notation /\ A for the greatest lower bound of A and the notation Y A for the 
least upper bound of A. In computer science, some people also use |J A instead of V ^ 
and the symbol |J upside down instead of /\. When A = {a,b}, we write a Ab for 
/\{a, b} and a V 6 for \/{a, b}. The element a A 6 is called the meet of a and b and a V 6 
is the join of a and b. (Some computer scientists use a □ 6 for a A 6 and aUb for a V b.) 

5. Observe that if it exists, /\ = T, the greatest element of X and if its exists, \/ = ±, 
the least element of X. Also, if it exists, /\ X = ± and if it exists, \/ X = T. 

The reader should look at the posets in Figures 14.21 and 14.31 for examples of the above 
notions. 

For the sake of completeness, we state the following fundamental result known as Zorn's 
Lemma even though it is unlikely that we will use it in this course. Zorn's lemma turns out 
to be equivalent to the axiom of choice. For details and a proof, the reader is referred to 
Suppes [lO] or Enderton [H]. 

Theorem 4.1.3 (Zorn's Lemma) Given a poset, (X, <), if every nonempty chain in X has 
an upper-bound, then X has some maximal element. 
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When we deal with posets, it is useful to use functions that are order-preserving as defined 
next. 

Definition 4.1.4 Given two posets {X, <x) and {Y, <y), a function, / : X — ^ y, is mono- 
tonic (or order-preserving) iff for all a, 6 G X, 

if a<xb then /(a) <y f{b). 

4.2 Lattices and Tarski's Fixed Point Theorem 

We now take a closer look at posets having the property that every two elements have a 
meet and a join (a greatest lower bound and a least upper bound). Such posets occur a 
lot more than we think. A typical example is the power set under inclusion, where meet is 
intersection and join is union. 

Definition 4.2.1 A lattice is a poset in which any two elements have a meet and a join. A 
complete lattice is a poset in which any subset has a greatest lower bound and a least upper 
bound. 

According to part (5) of the remark just before Zorn's Lemma, observe that a complete 
lattice must have a least element, _L, and a greatest element, T. 

Remark: The notion of complete lattice is due to G. Birkhoff (1933). The notion of a lattice 
is due to Dedekind (1897) but his definition used properties (L1)-(L4) listed in Proposition 
14.2.21 The use of meet and join in posets was first studied by C. S. Peirce (1880). 

Figure 113] shows the lattice structure of the power set of {a, b, c}. It is actually a complete 
lattice. 



{a, b, c} 




Figure 4.4: The lattice 2^'^''''^> 
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It is easy to show that any finite lattice is a complete lattice and that a finite poset is a 
lattice iff it has a least element and a greatest element. 

The poset N+ under the divisibihty ordering is a lattice! Indeed, it turns out that the 
meet operation corresponds to greatest common divisor and the join operation corresponds 
to least common multiple. However, it is not a complete lattice. The power set of any set, 
X, is a. complete lattice under the subset ordering. Indeed, one will verify immediately that 
for any collection, C, of subsets of X, the least upper bound of C is its union, IJC, and 
the greatest lower bound of C is its intersection, f]C. The least element of 2^ is and its 
greatest element is X itself. 

The following proposition gathers some useful properties of meet and join. 

Proposition 4.2.2 If X is a lattice, then the following identities hold for all a,b,c E X : 

LI aVb^bVa, aAb^bAa 

L2 {aV b)V aV {bV c), (a A 6) A c = a A (6 A c) 

L3 aV a — a, a Aa ~ a 

LA (a V 6) A a = a, (a A 6) V a = a. 

Properties (LI) correspond to commutativity , properties (L2) to associativity , properties (L3) 
to idempotence and properties (L4) to absorption. Furthermore, for all a,b & X, we have 

a <b iff ay b = b iff a Ab = a, 

called consistency. 

Proof . The proof is left as an exercise to the reader. □ 

Properties (L1)-(L4) are algebraic properties that were found by Dedekind (1897). A 
pretty symmetry reveals itself in these identities: they all come in pairs, one involving A, 
the other involving V. A useful consequence of this symmetry is duality, namely, that each 
equation derivable from (L1)-(L4) has a dual statement obtained by exchanging the symbols 
A and V. What is even more interesting is that it is possible to use these properties to define 
lattices. Indeed, if X is a set together with two operations, A and V, satisfying (L1)-(L4), 
we can define the relation a<bhya\/b — b and then show that < is a partial order such 
that A and V are the corresponding meet and join. The first step is to show that 

ay b = b iff a Ab = a. 

li ay b — b, then substituting 6 for a V 6 in (L4), namely 

{a y b) A a = a, 

we get 

b Aa = a, 
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which, by (LI), yields 

a Ab = a, 

as desired. Conversely, if a A b = a, then by (LI) we have b A a = a, and substituting a for 
6 A a in the instance of (L4) where a and b are switched, namely 

(6 A a) V 6 = 6, 

we get 

a V 6 = 6, 

as claimed. Therefore, we can define a<6asaV6 = 6or equivalently as a A 6 = a. After a 
little work, we obtain 

Proposition 4.2.3 Let X be a set together with two operations A and V satisfying the 
axioms (L1)-(L4) of proposition 14-2. 2\ If we define the relation < bya<bifja\/b = b 



(equivalently, a A b = a), then < is a partial order and {X, <) is a lattice whose meet and 
join agree with the original operations A and V. 

The following proposition shows that the existence of arbitrary least upper bounds (or 
arbitrary greatest lower bounds) is already enough ensure that a poset is a complete lattice. 

Proposition 4.2.4 Let {X, <) be a poset. If X has a greatest element, T , and if every 
nonempty subset. A, of X has a greatest lower bound, /\ A, then X is a complete lattice. 
Dually, if X has a least element, _L, and if every nonempty subset. A, of X has a least upper 
bound, y A, then X is a complete lattice 

Proof. Assume X has a greatest element, T, and that every nonempty subset. A, of X has 
a greatest lower bound, /\A. We need to show that any subset, S, of X has a least upper 
bound. As X has a greatest element, T, the set, U, of upper bounds of S is nonempty and 
so, m = /\U exists. We claim that /\U = \/ S, i.e, m is the least upper bound of 5*. First, 
note that every element of S* is a lower bound of U since U is the set of upper bounds of S. 
As m = /\ f/ is the greatest lower bound of U, we deduce that s < m for all s E S, i.e., m is 
an upper bound of S. Next, if b is any upper bound for S*, then b E U and as m is a lower 
bound of U (the greatest one), we have m < b, i.e., m is the least upper bound of S. The 
other statement is proved by duality. □ 

We are now going to prove a remarkable result due to A. Tarski (discovered in 1942, 
published in 1955). A special case (for power sets) was proved by B. Knaster (1928). First, 
we define fixed points. 

Definition 4.2.5 Let {X,<) be a poset and let /: X — X be a function. An element, 
X G X, is a fixed point of f (sometimes spelled fixpoint) iff 

/(x) = X. 

An element, x G X, is a least (resp. greatest) fixed point of f if it is a fixed point of / and if 
x < y (resp. y < x) for every fixed point y of f. 
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Fixed points play an important role in certain areas of mathematics (for example, topol- 
ogy, differential equations) and also in economics because they tend to capture the notion of 
stability or equilibrium. 

We now prove the following pretty theorem due to Tarski and then immediately proceed 
to use it to give a very short proof of the Schroder-Bernstein Theorem (Theorem I2.9.9I) . 

Theorem 4.2.6 (Tarski's Fixed Point Theorem) Let {X,<) be a complete lattice and let 
f : X X be any monotonic function. Then, the set, F, of fixed points of f is a complete 
lattice. In particular, f has a least fixed point, 

Xmin = /\{X e X I /(x) < X} 

and a greatest fixed point 

a^max = e X \ x < f{x)}. 

Proof . We proceed in three steps. 

Step 1 . We prove that a;niax is the largest fixed point of /. 

Since Xmax is an upper bound oi A = {x E X \ x < f{x)} (the smallest one), we have 
X < Xjnax for all X & A. By monotonicity of /, we get /(x) < /(x^ax) and since x G A, we 
deduce 

X < f{x) < /(xmax) for all X e A, 

which shows that /(x^ax) is an upper bound of A. As Xmax is the least upper bound of A, 
we get 

•^max — /"(-^max)- (*) 

Again, by monotonicity, from the above inequality, we get 

/(^^max) ^ /(/(^^max)); 

which shows that /(xmax) ^ ^- As Xmax is an upper bound of A, we deduce that 

/(s^max) ^ Xjxiax- (**) 

But then, (*) and (**) yield 

/'(•^max) '^max; 

which shows that Xmax is a fixed point of /. If x is any fixed point of /, that is, if /(x) = x, 
we also have x < /(x), i.e., x E A. As Xmax is the least upper bound of A, we have x < Xmax, 
which proves that Xmax is the greatest fixed point of /. 

Step 2. We prove that Xmin is the least fixed point of /. 

This proof is dual to the proof given in Step 1. 
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Step 3. We know that the set of fixed points, F, of / has a least element and a greatest 
element, so by Proposition 14.2.41 it is enough to prove that any nonempty subset, S F, 
has a greatest lower bound. If we let 

I = {x&X\x<s for all s G S" and x < f{x)}, 

then we claim that a = \/ / is a fixed point of / and that it is the greatest lower bound of S. 

The proof that a = \/ 1 is a. fixed point of / is analogous to the proof used in Step 1. 
Since a is an upper bound of /, we have x < a for all x E I. By monotonicity of / and the 
fact that a; G /, we get 

X < fix) < f{a). 

Thus, /(a) is an upper bound of / and so, as a is the least upper bound of /, we have 

a < f{a). (t) 

By monotonicity of /, we get /(a) < /(/(a)). Now, to claim that /(a) G /, we need to check 
that /(a) is a lower bound of S. However, by definition of /, every element of S is an upper 
bound of / and since a is the least upper bound of J, we must have a < s for all s G 5* i.e., 
a is a lower bound of S. By monotonicity of / and the fact that 5 is a set of fixed points, 
we get 

/(a) < f{s) = s, for all G S, 

which shows that /(a) is a lower bound of 5* and thus, /(a) G /, as contended. As a is an 
upper bound of / and /(a) G /, we must have 

/(«) < a, (tt) 
and together with (f), we conclude that /(a) = a, i.e., a is a fixed point of /. 

Since we already proved that a is a lower bound of S it only remains to show that if x is 
any fixed point of / and a; is a lower bound of S, then x < a. But, if x is any fixed point of 
/ then X < f{x) and since x is also a lower bound of S, then x E I. As a is an upper bound 
of /, we do get x < a. □ 

It should be noted that the least upper bounds and the greatest lower bounds in F do 
not necessarily agree with those in X. In technical terms, F is generally not a sub lattice of 
X. 

Now, as promised, we use Tarski's Fixed Point Theorem to prove the Schroder-Bernstein 
Theorem. 

Theorem 12.9.91 Given any two sets, A and B, if there is an injection from A to B and 
an injection from B to A, then there is a bijection between A and B. 

Proof . Let f : A ^ B and g: B ^ Ahe two injections. We define the function, y?: 2"^ — » 2^, 

by 

^{S) = A-g{B-f{S)), 
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for any S C A. Because of the two complementations, it is easy to check that (p is monotonic 
(ckeck it). As 2^ is a complete lattice, by Tarski's fixed point theorem, the function has 
a fixed point, that is, there is some subset C C A so that 

C = A-g{B-f{C)). 

By taking the complement of C in A, we get 

A-C = g{B-f{C)). 

Now, as / and g are injections, the restricted functions / \ C: C f{C) and 

g \ {B — f{C)) : {B — /(C)) ^ {A — C) are bijections. Using these functions, we define the 

function, h: A B, as follows: 



h{a) 



f{a) if a G C 

{g\{B-f{C))-\a) liaiC. 



The reader will check that h is indeed a bijection. □ 

The above proof is probably the shortest known proof of the Schroder-Bernstein Theorem 
because it uses Tarski's fixed point theorem, a powerful result. If one looks carefully at the 
proof, one realizes that there are two crucial ingredients: 

1. The set C is closed under (70/, that is, g o f{C) C C. 

2. A-C C g{B). 

Using these observations, it is possible to give a proof that circumvents the use of Tarski's 
theorem. Such a proof is given in Enderton [H], Chapter 6, and we give a sketch of this 
proof below. 

Define a sequence of subsets, C„, of A by recursion as follows: 

Co = A-g{B) 

Cn+i = {gof){C^), 

and set 

C=[jCn. 

Clearly, A — C C g{B) and since direct images preserve unions, {g o f){C) C C. The 
definition of h is similar to the one used in our proof: 



h{a) 



f{a) if a G C 

ig\iA-C))-\a) iia^C. 



When a ^ C, i.e., a G A — C, as A — C C g{B) and g is injective, g ^(a) is indeed well-defined. 
As / and g are injective, so is g^^ on A — C. So, to check that h is injective, it is enough 
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to prove that /(a) = g^^ih) with a G C and b ^ C is impossible. However, if /(a) = g^^{b), 
then {g o /)(a) = b. Since {g o /)(C) C C and a G C, we get b = {g o f){a) G C, yet 6 ^ C, 
a contradiction. It is not hard to verify that h is surjective and therefore, h is a bijection 
between A and B. □ 

The classical reference on lattices is Birkhoff [6]. We highly recommend this beautiful 
book (but it is not easy reading!). 

We now turn to special properties of partial orders having to do with induction. 

4.3 Weil-Founded Orderings and Complete Induction 

Have you ever wondered why induction on N actually "works"? The answer, of course, is 
that N was defined in such a way that, by Theorem 11.8.41 it is the "smallest" inductive set! 
But this is not a very illuminating answer. The key point is that every nonempty subset of 
N has a least element. This fact is intuitively clear since if we had some nonempty subset of 
N with no smallest element, then we could construct an infinite strictly decreasing sequence, 
ko > ki > ■ ■ ■ > kn > ■ ■ ■ ■ But this is absurd, as such a sequence would eventually run into 
and stop. It turns out that the deep reason why induction "works" on a poset is indeed that 
the poset ordering has a very special property and this leads us to the following definition: 

Definition 4.3.1 Given a poset, {X, <), we say that < is a well-order (well ordering) and 
that X is well-ordered by < iff every nonempty subset of X has a least element. 

When X is nonempty, if we pick any two-element subset, {a,b}, of X, since the subset 
{a,b} must have a least element, we see that either a < b or b < a, i.e., every well-order is 
a total order. First, let us confirm that N is indeed well-ordered. 

Theorem 4.3.2 (Well- Ordering ofN) The set of natural numbers, N, is well-ordered. 

Proof . Not surprisingly we use induction, but we have to be a little shrewd. Let A be any 
nonempty subset of N. We prove by contradiction that A has a least element. So, suppose 
A does not have a least element and let P{m) be the predicate 

P(m) = (VA; G N){k <m^ k^A), 

which says that no natural number strictly smaller than m is in A. We will prove by induction 
on m that P{m) holds. But then, the fact that P{m) holds for all m shows that A = 0, a 
contradiction. 

Let us now prove P{m) by induction. The base case P{0) holds trivially. Next, assume 
P{m) holds; we want to prove that P{m -\- 1) holds. Pick any k < m -\- 1. Then, either 

(1) A; < m, in which case, by the induction hypothesis, k ^ A; or 



128 



CHAPTER 4. PARTIAL ORDERS AND EQUIVALENCE RELATIONS 



(2) k = m. By the induction hypothesis, P{m) holds. Now, if m was in A, as P{m) holds 
no k < m would belong to A and m would be the least element of A, contradicting the 
assumption that A has no least element. Therefore, m ^ A. 

Thus, in both cases, we proved that if < m + 1, then k ^ A, establishing the induction 
hypothesis. This concludes the induction and the proof of Theorem I4.3.2[ □ 

Theorem 14.3.21 yields another induction principle which is often more flexible that our 
original induction principle. This principle, called complete induction (or sometimes strong 
induction) , is stated below. Recall that N+ = N — {0}. 

Complete Induction Principle on N. 

In order to prove that a predicate, P{n), holds for all n G N it is enough to prove that 

(1) -P(O) holds (the base case) and 

(2) for every m G N+, if (V/e G N){k < m ^ P{k)) then P(m). 
As a formula, complete induction is stated as 

P(0) A (Vm G N+)[(VA; G N)(A; < m ^ P{k)) P{m)] (Vn G N)P(n). 

The difference between ordinary induction and complete induction is that in complete 
induction, the induction hypothesis, (VA; G N)(A; < m ^ P{k)), assumes that P{k) holds for 
all k < m and not just for m — 1 (as in ordinary induction), in order to deduce P{m). This 
gives us more proving power as we have more knowledge in order to prove P{m). 

We will have many occasions to use complete induction but let us first check that it is 
a valid principle. Even though we will give a more general proof of the validity of complete 
induction for a well-ordering, we feel that it will be helpful to give the proof in the case of 
N as a warm-up. 

Theorem 4.3.3 The complete induction principle for N is valid. 

Proof . Let P{n) be a predicate on N and assume that P{n) satisfies conditions (1) and (2) 
of complete induction as stated above. We proceed by contradiction. So, assume that P{n) 
fails for some n G N. If so, the set 

F = {n G N I P{n) = false} 

is nonempty. By Theorem 14. 3. 2^ the set A has a least element, m, and thus, 

P(m) = false. 

Now, we can't have m = 0, as we assumed that P(0) holds (by (1)) and since m is the least 
element for which P{m) = false, we must have 



P{k) = true for all k < m. 
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But, this is exactly the premise in (2) and as we assumed that (2) holds, we deduce that 

P(m) = true, 

contradicting the fact that we already know that P{m) — false. Therefore, P(n) must hold 
for all n e N. □ 

Reiiicirk: In our statement of the principle of complete induction, we singled out the base 

case, (1), and consequently, wc stated the induction step (2) for every m G N+, excluding 
the case m = 0, which is already covered by the base case. It is also possible to state the 
principle of complete induction in a more concise fashion as follows: 

(Vm e N)[{Vk eN){k<m^ P{k)) => P{m)] => (Vn e N)P(n). 

In the above formula, observe that when m — 0, which is now allowed, the premise 
(V/c G < m ^ -P(^)) of the implication within the brackets is trivially true and so, 

P(0) must still be established. In the end, exactly the same amount of work is required but 
some people prefer the second more concise version of the principle of complete induction. 
We feel that it would be easier for the reader to make the transition from ordinary induction 
to complete induction if we make explicit the fact that the base case must be established. 

Let us illustrate the use of the complete induction principle by proving that every natural 
number factors as a product of primes. Recall that for any two natural numbers, a, 6 G N 
with 6 7^ 0, we say that b divides a iff a = bq, for some g G N. In this case, we say that a is 
divisible by b and that b is a factor of a. Then, wc say that a natural number, p G N, is a 
prime number (for short, a prime) if p > 2 and if p is only divisible by itself and by 1. Any 
prime number but 2 must be odd but the converse is false. For example, 2, 3, 5, 7, 11, 13, 17 
are prime numbers, but 9 is not. It can be shown that there are infinitely many prime 
numbers but to prove this, we need the following Theorem: 

Theorem 4.3.4 Every natural number, n > 2 can be factored as a product of primes, that 
is, n can be written as a product, n — p^^ ■ • ■p^'^ , where the piS are pairwise distinct prime 
numbers and > 1 (1 < i < k). 

Proof. We proceed by complete induction on n > 2. The base case, n = 2 is trivial, since 2 
is prime. 

Consider any n > 2 and assume that the induction hypothesis holds, that is, every m 
with 2 < m < n can be factored as a product of primes. There are two cases: 

(a) The number n is prime. Then, we are done. 

(b) The number n is not a prime. In this case, n factors as n = nin2, where 2 < ni,n2 < n. 
By the induction hypothesis, ni has some prime factorization and so does n2. If 
{pi, . . . ,pk} is the union of all the primes occurring in these factorizations of rii and 
712, we can write 

rii = p\^ ■■ -Pk and n2 = p{^ ■ ■ -pk, 
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where ih,jh > and, in fact, ih + jh > 1, for 1 < /i < A;. Consequently, n factors as 
the product of primes, 

*i + 7l ik+ih 

n = Pi ■ ■ -p^ , 
with ih+ jh > 1, estabhshing the induction hypothesis. □ 

Remark: It can be shown that the prime factorization of a natural number is unique up 
to permutation of the primes pi, . . . ,pk but this requires the Euchdean Division Lemma. 
However, we can prove right away that there are infinitely primes. 

Theorem 4.3.5 Given any natural number, n > 1, there is a prime number, p, such that 
p > n. Consequently, there are infinitely many primes. 

Proof . Let m = n! + 1. If m is prime, we are done. Otherwise, by Theorem 14. 3. 4[ the number 
m has a prime decomposition. We claim that p > n for every prime in this decomposition. 
If not, 2 < p < n and then p would divide both n\ + 1 and n\, so p would divide 1, a 
contradiction. □ 

As an application of Theorem 14.3.21 we prove the "Euclidean Division Lemma" for the 
integers. 

Theorem 4.3.6 (Euclidean Division Lemma for %) Given any two integers, a,b ^ Z, with 
b ^ 0, there is some unique integer, g G Z (the quotient), and some unique natural number, 
r e N (the remainder or residue), so that 

a = bq + r with < r < \b\. 

Proof. First, let us prove the existence of q and r with the required condition on r. We 
claim that if we show existence in the special case where a, 6 G N (with b ^ 0), then we can 
prove existence in the general case. There are four cases: 

1. If a, 6 G N, with b ^ 0, then we are done. 

2. If a > and 6 < 0, then —b > 0, so we know that there exist q, r with 

a = {—b)q + r with < r < — 6 — 1. 

Then, 

a = b{—q) + r with < r < |6| — 1. 

3. If a < and 6 > 0, then —a > 0, so we know that there exist q, r with 

—a = bq + r with < r < 6 — 1. 

Then, 

a = b{~q) — r with < r < 6 — 1. 

If r = 0, we are done. Otherwise, 1 < r < 6 — 1, which implies 1<6 — r<6 — 1, so 
we get 

a = b{-q)-b + b-r = b{-{q + l)) + b-r with <b-r <b-l. 
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4. If a < and 6 < 0, then —a > and —b > 0, so we know that there exist g, r with 

—a = {—b)q + r with < r < — 6 — 1. 

Then, 

a = bq — r with < r < — 6 — 1. 

If r = 0, we are done. Otherwise, 1 < r < — 6 — 1, which implies l<—b — r<—b — l, 
so we get 

a = bq + b — b — r = b{q + 1) + (—6 — r) with < — 6 — r < |6| — 1. 

We are now reduced to proving the existence of q and r when a, 6 G N with b ^ 0. Consider 
the set 

R = {a~bq eN\ q EN}. 

Note that a G -R, by setting q = 0, since a G N. Therefore, R is nonempty. By Theorem 
I4.3.2[ the nonempty set, R, has a least element, r. We claim that r < b—1 (of course, r > 
as -R C N). If not, then r > b, and so r — 6 > 0. As r G -R, there is some g G N with 
r = a — bq. But now, we have 

r — b = a — bq — b = a — b{q + 1) 

and as r — 6 > 0, we see that r — b E R with r — b < r (since 6 7^ 0), contradicting the 
minimality of r. Therefore, < r < 6 — 1, proving the existence of q and r with the required 
condition on r. 

We now go back to the general case where a, 6 G Z with 6 7^ and we prove uniqueness 
of q and r (with the required condition on r). So, assume that 

a = bqi + ri = bq2 + 7*2 with < ri < |6| — 1 and < r2 < |6| — 1. 

Now, as < ri < — 1 and < r2 < |&| — 1, we have |ri— r2| < \b\, and from bqi+ri = bq2+r2, 
we get 

KQ2 - qi) = n -r2, 

which yields 

\b\\Q2 - qi\ = \ri - r2|. 

Since |ri — < we must have ri = r2. Then, from 6(^2 — qi) = ri — r2 = 0, as 6 7^ 0, 
we get qi = q2, which concludes the proof. □ 

We will now show that complete induction holds for a very broad class of partial orders 
called well-founded orderings that subsume well-orderings. 

Definition 4.3.7 Given a poset, (X, <), we say that < is a well-founded ordering (order) 
and that X is well-founded iff X has no infinite strictly decreasing sequence 

Xq > Xi > X2 > ■ ■ ■ > Xn > Xn+l > ■ ■ ■ . 
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The following property of well-founded sets is fundamental: 

Proposition 4.3.8 A poset, {X,<), is well-founded iff' every nonempty subset of X has a 
minimal element. 

Proof. First, assume that every nonempty subset of X has a minimal element. If we had an 
infinite strictly decreasing sequence, xq > xi > X2 > ■ ■ ■ > Xn > ■ ■ ■ , then the set A = {xn} 
would have no minimal element, a contradiction. Therefore, X is well-founded. 

Now, assume that X is well-founded. We prove that A has a least element by contradic- 
tion. So, let A be some nonempty subset of X and suppose A has no least element. This 
means that for every a G A, there is some b E A with a > b. Using the Axiom of Choice 
(Graph Version), there is some function, g: A A, with the property that 

a > g{a), for all a E A. 

Now, since A is nonempty, we can pick some element, say a E A. By the recursion Theorem 
(Theorem 12.5.11) . there is a unique function, / : N — >• A, so that 

/(O) = a, 
/(n + 1) = g{f{n)) for all G N. 

But then, / defines an infinite sequence, {x„}, with x„ = f{n), so that Xn > Xn+i for all 
G N, contradicting the fact that X is well-founded. □ 

So, the seemingly weaker condition that there is no infinite strictly decreasing sequence 
in X is equivalent to the fact that every nonempty subset of X has a minimal element. If 
X is a total order, any minimal element is actually a least element and so, we get 

Corollary 4.3.9 A poset, {X, <), is well-ordered iff < is total and X is well-founded. 

Note that the notion of a well-founded set is more general than that of a well-ordered 
set, since a well-founded set is not necessarily totally ordered. 

Remark: Suppose we can prove some property, P, by (ordinary) induction on N. Then, I 
claim that P can also be proved by complete induction on N. To see this, observe first that 
the base step is identical. Also, for all m G N+, the implication 

(Vfc G N)(A; < m ^ P{k)) P{m - 1) 

holds and since the induction step (in ordinary induction) consists in proving for all m G 
that 

P(m- 1) ^ P(m) 

holds, from this implication and the previous implication we deduce that for all m G N+, 
the implication 

iy/k G n){k < m ^ P{k)) =^ P{m) 
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holds, which is exactly the induction step of the complete induction method. So, we see that 
complete induction on N implies ordinary induction on N. The converse is also true but we 
leave it as a fun exercise. But now, by Theorem 14.3.21 (ordinary) induction on N implies that 
N is well-ordered and by Theorem I4.3.3[ the fact that N is well-ordered implies complete 
induction on N. Since we just showed that complete induction on N implies (ordinary) 
induction on N, we conclude that all three are equivalent, that is 

(ordinary) induction on N is valid 
iff 

complete induction on N is valid 
iff 

N is well-ordered. 

These equivalences justify our earlier claim that the ability to do induction hinges on some 
key property of the ordering, in this case, that it is a well-ordering. 

We finally come to the principle of complete induction (also called transfinite induction 
or structural induction ), which, as we shall prove, is valid for all well-founded sets. Since 
every well-ordered set is also well-founded, complete induction is a very general induction 
method. 

Let {X,<) be a well-founded poset and let P be a predicate on X (i.e., a function 
P: X ^ {true, false}). 

Principle of Complete Induction on a Well-Founded Set. 

To prove that a property P holds for all z G X, it suffices to show that, for every x & X, 

{*) if X is minimal or P{y) holds for all y < x, 
(**) then P{x) holds. 

The statement (*) is called the induction hypothesis , and the implication 

for all X, (*) implies (**) is called the induction step. Formally, the induction principle 
can be stated as: 

(Vx G X)[(Vy eX){y<x^ P{y)) =^ P{x)] (V^ G X)P{z) (CI) 

Note that if x is minimal, then there is no ?/ G X such that y < x, and 

{Wy E X){ii < X => P{y)) is true. Hence, we must show that P{x) holds for every minimal 

element, x. These cases are called the base cases. 

Complete induction is not valid for arbitrary posets (see the problems) but holds for 
well-founded sets as shown in the following theorem. 

Theorem 4.3.10 The principle of complete induction holds for every well-founded set. 
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Proof. We proceed by contradiction. Assume that (CI) is false. Then, 

(Va; e X)[(Wy G X){y <x^ P{y)) P{x)] (1) 

holds and 

(V^ G X)P{z) (2) 
is false, that is, there is some z G X so that 

P{z) = false. 

Hence, the subset F of X defined by 

F = {xeX \ P{x) = false} 

is nonempty. Since X is well founded, by Proposition 14.3.81 F has some minimal element, b. 
Since (1) holds for all x G X, letting x = b, we see that 

[{\/yeX)iy<b^Piy))^Pib)] (3) 

holds. If b is also minimal in X, then there is no ?/ G X such that y < b and so, 

(yyeX){y<b^P{y)) 

holds trivially and (3) implies that P{b) = true, which contradicts the fact that b E F. 
Otherwise, for every y E X such that y < b, P{y) = true, since otherwise y would belong 
to F and b would not be minimal. But then, 

iWyeX){y<b^P{y)) 

also holds and (3) implies that P{b) = true, contradicting the fact that b E F. Hence, 
complete induction is valid for well-founded sets. □ 

As an illustration of well-founded sets, we define the lexicographic ordering on pairs. 
Given a partially ordered set (X, <), the lexicographic ordering, <<, on X x X induced by 
< is defined a follows: For all x, y, x', y' G X, 

{x,y) « {x',y') iff either 

X = x' and y = y' or 

X < x' or 

X = x' and y < y'. 

We leave it as an exercise to check that << is indeed a partial order on X x X. The 
following proposition will be useful. 
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Proposition 4.3.11 // {X, <) is a well-founded set, then the lexicographic ordering « on 
X X X is also well founded. 

Proof . We proceed by contradiction. Assume that there is an infinite decreasing sequence 
{{xi,yi))i in X X X. Then, either, 

(1) There is an infinite number of distinct Xi, or 

(2) There is only a finite number of distinct Xj. 

In case (1), the subsequence consisting of these distinct elements forms a decreasing 
sequence in X, contradicting the fact that < is well founded. In case (2), there is some k 
such that Xi — Xj+i, for all i > k. By definition of <<, the sequence {yi)i>k is a decreasing 
sequence in X, contradicting the fact that < is well founded. Hence, << is well founded on 
X xX.n 

As an illustration of the principle of complete induction, consider the following example 
in which it is shown that a function defined recursively is a total function. 

Example (Ackermann's function) The following function, A: N x N — > N, known as 
Ackermann's function is well known in recursive function theory for its extraordinary rate 
of growth. It is defined recursively as follows: 

A{x, y) — if X = then y + 1 

else if y = then A{x — 1,1) 
else A{x - 1, A{x, y-l)). 

We wish to prove that A is a total function. We proceed by complete induction over the 
lexicographic ordering on N x N. 

1. The base case is a; = 0, y = 0. In this case, since /1(0,|/) = y + 1, yl(0, 0) is defined 
and equal to 1. 

2. The induction hypothesis is that for any (m, n), A{m' ,n') is defined for all 
{m',n') « {m,n), with {m,n) ^ {m',n'). 

3. For the induction step, we have three cases: 

(a) If m = 0, since A(0, y) — y A(0, n) is defined and equal to n + 1. 

(b) If m 7^ and n = 0, since (m — 1, 1) << (m, 0) and (m —1,1)7^ (m, 0), by the 
induction hypothesis, A{m — 1, 1) is defined, and so A{m, 0) is defined since it is 
equal to A{m — 1, 1). 
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(c) If m 7^ and n ^ 0, since (m, n — 1) << {m,n) and {m,n — 1) ^ {m,n), by 
the induction hypothesis, A{m, n — 1) is defined. Since (m — 1, y) « (m, z) and 

{m — l,y) ^ (m. z) no matter what y and z are, 

(m — l,A{m,n — 1)) << {m,n) and (m — l,A(m, n — 1)) 7^ {m,n), and by 
the induction hypothesis, 74(m — l,A{m,n — 1)) is defined. But this is precisely 
A{m,n), and so A{m,n) is defined. This concludes the induction step. 

Hence, A{x, y) is defined for all x, y > 0. □ 

4.4 Unique Prime Factorization in Z and GCD's 

In the previous section, we proved that every natural number, n > 2, can be factored as a 
product of primes numbers. In this section, we use the Euclidean Division Lemma to prove 
that such a factorization is unique. For this, we need to introduce greatest common divisors 
(gcd's) and prove some of their properties. 

In this section, it will be convenient to allow to be a divisor. So, given any two integers, 
a, 6 G Z, we will say that b divides a and that a is a multiple of b iS a = bq, for some q' e Z. 
Contrary to our previous definition, 6 = is allowed as a divisor. However, this changes very 
little because if divides a, then a = Og = 0, that is, the only integer divisible by is 0. 

We begin by introducing a very important notion in algebra, that of an ideal, and prove 
a fundamental property of the ideals of Z. 

Definition 4.4.1 An ideal ofZ is any nonempty subset, 3, of Z satisfying the following two 
properties: 

(IDl) If a, 6 e 3, then b-ae3. 

(ID2) If a e a, then ak e J for every k e Z. 

An ideal, 3, is a principal ideal if there is some a e !J, called a generator, such that 

3 = {ak I A; G Z}. The equality 3 = {ak | A; G Z} is also written as Of = aZ or as 3 — (a). 

The ideal 3 = {0) = {0} is called the null ideal. 

Note that if 3 is an ideal, then J = Z iff 1 G 3. Since by definition, an ideal 3 is nonempty, 
there is some a & 3, and by (IDl) we get — a — a & 3. Then, for every a G CJ, since G J, 
by (IDl) we get -a G 3. 

Theorem 4.4.2 Every ideal, 3, of 1., is a principal ideal, i.e., 3 = roL for some unique 
men, with m > iff 3 ^ (0). 

Proof . Note that J = (0) iff 2^ = OZ and the theorem holds in this case. So, assume that 
3 ^ (0). Then, our previous observation that —a G 3 for every a G J implies that some 
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positive integer belongs to 3 and so, the set 3 fl N+ is nonempty. As N is well-ordered, this 
set has a smallest element, say m > 0. We claim that 3 = mZ. 

As m G Df, by (ID2), mZ C 3. Conversely, pick any n & 3. By the Euclidean division 
Theorem, there are unique g G Z and r G N so that n = mq + r, with < r < m. If 
r > 0, since m E 3, by (ID2), mq G 3 and by (IDl), we get r = n — mq G 3. Yet r < m, 
contradicting the minimality of m. Therefore, r = 0, so = mq G mZ, establishing that 
3 C mZ and thus, 3 = mZ, as claimed. As to uniqueness, clearly (0) 7^ mZ if m 7^ 0, so 
assume mZ = m'Z, with m > and m' > 0. Then, m divides m' and m' divides m, but we 
already proved earlier that this implies m = m'. □ 

Theorem 14.4.21 is often phrased: Z is a principal ideal domain, for short, a PID. Note 
that the natural number m such that 3 = mZ is a divisor of every element in 3. 

Corollary 4.4.3 For any two integers, a,h E Z, there is a unique natural number, c? G N, 
and some integers, m, f G Z, so that d divides both a and b and 

ua + vb = d. 

(The above is called the Bezout identity.) Furthermore, = iff' a = and 6 = 0. 

Proof . It is immediately verified that 

3 = {ha + kb\h,k eZ} 

is an ideal of Z with a,b E 3. Therefore, by Theorem I4.4.2[ there is a unique, (i G N, so that 
3 = dZ. We already observed that d divides every number in 3 so, as a, 6 G 3, we see that 
d divides a and b. li d = 0, as d divides a and b, we must have a = b = 0. Conversely, if 
a = b = 0, then d = ua + bv = 0. □ 

The natural number d of corollary 14.4.31 divides both a and b. Moreover, every divisor of 
a and b divides d = ua + vb. This motivates the definition: 

Definition 4.4.4 Given any two integers, a, 6 G Z, an integer, c? G Z, is a greatest common 
divisor of a and b (for short, a gcd of a and b) if d divides a and b and, for any integer, /i G Z, 
if h divides a and 6, then h divides d. We say that a and b are relatively prime if 1 is a gcd 
of a and b. 

Remarks: 

1. Assume a = 6 = 0. Then, any integer, c? G Z, is a divisor of 0. In particular, divides 
0. According to Definition SXll this implies gcd(0,0) = 0. If (a, 6) 7^ (0,0), then 1 
divides a and 6, so gcd(a, b) = d > In this case, if d' is any other gcd of a and 6, 
then d = qd' and d' = q'd for some q, q' G Z. So, d = qq'd which implies qq' = 1 (since 
d ^ 0) and thus, d' = ±d. So, according to the above definition, when {a,b) 7^ (0,0), 
gcd's are not unique. However, exactly one of d or —d is positive, so we will refer to 
this positive gcd as "the" gcd of a and b and write d = gcd (a, b). 
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2. Observe that d = gcd(a, b) is indeed the largest positive common divisor of a and b 
since every divisor of a and b must divide d. However, we did not use this property 
as one of the conditions for being a gcd because such a condition does not generahze 
to other rings where a total order is not available. Another minor reason is that if we 
had used in the definition of a gcd the condition that gcd(a, b) should be the largest 
common divisor of a and b, as every integer divides 0, gcd(0, 0) would be undefined! 

3. Our definition of the gcd makes sense even if we allow a = b = 0. In this case, 
gcd(0, 0) = 0. If we did not allows to be a divisor, the situation would be different. 
Indeed, if we had gcd(0, 0) = d for some d > 0, as every other positive integer, d', 
divides 0, every integer d' would have to divide d, which is absurd. This is why we 
relaxed our definition to allow to be a divisor. Nevertheless, the cases where a = 
or 6 = are somewhat degenerate cases so we prefer to stick to the simpler situation 
where we only consider gcd's for two nonzero integers. 

Let p G N be a prime number. Then, note that for any other integer, n, if p does not 
divide n, then gcd(p, n) = 1, as the only divisors of p are 1 and p. 

Proposition 4.4.5 Given any two integers, a,b E 'L, a natural number, d E N, is the 
greatest common divisor of a and b iff d divides a and b and if there are some integers, 
M, f G Z, so that 

ua + vb = d. (Bezout Identity) 

In particular, a and b are relatively prime iff there are some integers, u,v E Z, so that 

ua + vb = 1. (Bezout Identity) 

Proof. We already observed that half of Proposition 14.4.51 holds, namely if c? G N divides a 
and b and if there are some integers, m, f G Z, so that ua + vb = d, then, d is the gcd of a 
and b. Conversely, assume that d = gcd(a, b). If d = 0, then a = b = and the proposition 
holds trivially. So, assume d > 0, in which case (a, &) 7^ (0,0). By Corollary I4.4.3[ there is 
a unique m G N with m > that divides a and b and there are some integers, m, G Z, so 
that 

ua + vb = m. 

But now, m is also the (positive) gcd of a and b, so d = m and our Proposition holds. Now, 
a and b are relatively prime iff gcd(a, 6) = 1 in which case the condition that d = 1 divides 
a and b is trivial. □ 

Remark: The gcd of two nonzero integers can be found using a method involving Euclidean 
division and so can the numbers u and v. 

Proposition 14.4.51 implies a very crucial property of divisibility in any PID. 

Proposition 4.4.6 (Euclid's proposition) Let a,b,c E Z be any integers. If a divides be and 
a is relatively prime to b, then a divides c. 
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Proof. From Proposition 14.4.51 a and b are relatively prime iff there exist some integers, 
u,v E Z such that 

ua + vb = 1. 

Then, we have 

uac + vbc = c, 

and since a divides be, it divides both uac and vbc and so, a divides c. □ 

In particular, if p is a prime number and if p divides ab, where a,b E Z are nonzero, then 
either p divides a or p divides b since if p does not divide a, by a previous remark, then p 
and a are relatively prime, so Proposition 14.4.61 implies that p divides c. 

Proposition 4.4.7 Let a,bi, . . . ,bm € Z be any integers. If a and bi are relatively prime 
for all i, with 1 < i < m, then a and &i ■ ■ ■ 6m Q'^e relatively prime. 

Proof. We proceed by induction on m. The case m = 1 is trivial. Let c = 62 ■ ■ "^m- By 
the induction hypothesis, a and c are relatively prime. Let d the gcd of a and 61 c. We 
claim that d is relatively prime to bi. Otherwise, d and 61 would have some gcd di ^ 1 
which would divide both a and 61, contradicting the fact that a and bi are relatively prime. 
Now, by Proposition I4.4.6[ since d divides bic and d and bi are relatively prime, d divides 
c = 62 ■ ■ ■ bm- But then, d is a divisor of a and c, and since a and c are relatively prime, 
d = 1, which means that a and 61 ■ ■ ■ &m are relatively prime. □ 

We can now prove the uniqueness of prime factorizations in N. divisor to be 0. 

Theorem 4.4.8 (Unique Prime Factorization in N) For every nonzero natural number, 
a > 2, there exists a unique set, {{pi, ki), . . . , {pm, km)}, where the pi 's are distinct prime 
numbers and the ki's are (not necessarily distinct) integers, with m>l,ki>l, and 

a=Pi'---pt- 

Proof. The existence of such a factorization has already been proved in Theorem 14. 3. 4[ 
Let us now prove uniqueness. Assume that 

a=Pi'---pt and a = g^---g^\ 

Thus, we have 

pt'-'-pt =(ll'■■■(lt■ 
We prove that m = n, pi = Qi and hi = ki, for all i, with 1 < i < n. The proof proceeds by 
induction on hi + ■■■ + hn- 

If hi + ■ ■ ■ + hn = 1, then n = 1 and hi = 1. Then, 
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and since qi and the Pi are prime numbers, we must have m = 1 and pi = q\ (a prime is 
only divisible by 1 or itself). 

If /ii + ■ ■ ■ + /i„ > 2, since hi > 1, we have 

Pi' ■■■Pt = (li(l, 

with 

q — ll In ^ 

where {hi — 1) + ■ ■ ■ + hn > 1 (and gf^"^ = 1 if /ii = 1). Now, if qi is not equal to any 
of the pj, by a previous remark, qi and Pi are relatively prime, and by Proposition 14.4.71 qi 
and pI^ ■ ■ -p^ are relatively prime. But this contradicts the fact that qi divides ■ ■ -p^. 
Thus, qi is equal to one of the Pi. Without loss of generality, we can assume that qi = pi- 
Then, as gi 7^ 0, we get 

n'^i^l . . . T)'^™ — n^^~^ . . . n^" 
Pi Pm ~ HI Hn •! 

where p^"^ = 1 if A;i = 1, and gj*'"^ = li{hi = l. Now, (/ii - 1) H h /i„ < /^i H h /i„, 

and we can apply the induction hypothesis to conclude that m = n, pi = qi and hi = ki, 
with 1 < i < n. □ 

Theorem 14.4.81 is a basic but very important result of number theory and it has many 
applications. It also reveals the importance of the primes as the building blocks of all 
numbers. 

Remark: Theorem 14.4.81 also applies to any nonzero integer a G Z — {—1, +1}, by adding a 
suitable sign in front of the prime factorization. That is, we have a unique prime factorization 
of the form 

a = ±p^^' ■■■p'^. 

Theorem 14.4.81 shows that Z is a unique factorization domain, for short, a UFD. Such rings 
play an important role because every nonzero element which is not a unit (i.e., which is 
not invertible) has a unique factorization (up to some unit factor) into so-called irreducible 
elements which generalize the primes. 

We now take a well-deserved break from partial orders and induction and study equiva- 
lence relations, an equally important class of relations. 

4.5 Equivalence Relations and Partitions 

Equivalence relations basically generalize the identity relation. Technically, the definition of 
an equivalence relation is obtained from the definition of a partial order (Definition I4.1.ip 
by changing the third condition, antisymmetry, to symmetry. 

Definition 4.5.1 A binary relation, R, on a set, X, is an equivalence relation iff it is 
reflexive, transitive and symmetric, that is: 
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(1) {Reflexivity): aRa, for all a & X; 

(2) (Transitivity): If aRb and bRc, then aRc, for all a,b,c E X. 

(3) [symmetry): If ai?6, then 6i?a, for all a,b E X. 

Here are some examples of equivalence relations. 

1. The identity relation, idx, on a set X is an equivalence relation. 

2. The relation X x X is an equivalence relation. 

3. Let S be the set of students in CSE260. Define two students to be equivalent iff they 
were born the same year. It is trivial to check that this relation is indeed an equivalence 
relation. 

4. Given any natural number, p > 1, define a relation on Z as follows: 

m = n (mod p) 

iff p I m — n, i.e., p divides m — n. It is an easy exercise to check that this is indeed an 
equivalence relation called congruence modulo p. 

5. Equivalence of propositions is the relation defined so that P = Q iff P =^ Q and 
Q =^ P are both provable (say, classically) . It is easy to check that logical equivalence 
is an equivalence relation. 

6. Suppose / : X — > y is a function. Then, we define the relation =y on X by 

x=fy iff f{x) = f{y). 

It is immediately verified that =/ is an equivalence relation. Actually, we are going 
to show that every equivalence relation arises in this way, in terms of (surjective) 
functions. 

The crucial property of equivalence relations is that they partition their domain, X, into 
pairwise disjoint nonempty blocks. Intuitively, they carve out X into a bunch of puzzle 
pieces. 

Definition 4.5.2 Given an equivalence relation, R, on a set, X, for any x & X, the set 

[x]r = {yeX\ xRy} 

is the equivalence class ofx. Each equivalence class, [x]ji, is also denoted xr and the subscript 
R is often omitted when no confusion arises. The set of equivalence classes of R is denoted 
by X/R. The set X/R is called the quotient of X by R or quotient of X modulo R. The 
function, tt: X — > X/R, given by 

tt{x) = [x]r, X e X, 
is called the canonical projection (or projection) of X onto X/ R. 
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Since every equivalence relation is reflexive, i.e., xRx for every x G X, observe that 
a; G [x][i for any a; G -R, that is, every equivalence class is nonempty . It is also clear that the 
projection, tt: X ^ X/R, is surjective. The main properties of equivalence classes are given 
by 

Proposition 4.5.3 Let R be an equivalence relation on a set, X. For any two elements 
X, y G X, we have 

xRy iff' [x] = [y]. 
Moreover, the equivalences classes of R satisfy the following properties: 

(1) [x] ^ 0, for allxe X; 

(2) If [x] ^ [y] then [x] n [y] = 0; 

(3) X = \J^^^[x]. 

Proof . First, assume that [x] = [y]. We observed that by reflexivity, ?/ G [y]. As [x] = [y], 
we get y G [x] and by definition of [x], this means that xRy. 

Next, assume that xRy. Let us prove that [y] C [x]. Pick any z E [y]; this means that 
yRz. By transitivity, we get xRz, ie., z E [x], proving that [y] C [x]. Now, as R is symmetric, 
xRy implies that yRx and the previous argument yields [x] C [y]. Therefore, [x] = [y], as 
needed. 

Property (1) follows from the fact that x G [x] (by reflexivity). 

Let us prove the contrapositive of (2). So, assume [x] fl [y] ^ 0. Thus, there is some z so 
that z G \x\ and z E [y], i.e., 

xRz and yRz. 

By symmetry, we get zRy and by transitivity, xRy. But then, by the first part of the 
proposition, we deduce \x\ = [y], as claimed. 

The third property follows again from the fact that x G [x]. □ 

A useful way of interpreting Proposition 14.5.31 is to say that the equivalence classes of an 
equivalence relation form a partition, as defined next. 

Definition 4.5.4 Given a set, X, a partition of X is any family, 11 = of subsets of 

X such that 

(1) Xi 7^ 0, for alH G / (each Xi is nonempty); 

(2) If z 7^ j then Xi fl Xj = (the X, are pairwise disjoint); 

(3) X = IJ.gjXj (the family is exhaustive). 
Each set Xj is called a block of the partition. 
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In the example where equivalence is determined by the same year of birth, each equiva- 
lence class consists of those students having the same year of birth. Let us now go back to 
the example of congruence modulo p (with p > 0) and figure out what are the blocks of the 
corresponding partition. Recall that 

m = n (modp) 

iS m — n = pk for some G Z. By the division Theorem (Theorem 14.3.61) . we know that 
there exist some unique q, r, with m = pq + r and < r < p — 1. Therefore, for every m G Z, 

m = r (modp) with < r < p — 1, 

which shows that there are p equivalence classes, [0], [1], . . . , — 1], where the equivalence 
class, [r] (with < r < p — 1), consists of all integers of the form pq + r, where g G Z, i.e., 
those integers whose residue modulo p is r. 

Proposition 14.5.31 defines a map from the set of equivalence relations on X to the set of 
partitions on X. Given any set, X, let Equiv(X) denote the set of equivalence relations on 
X and let Part(X) denote the set of partitions on X. Then, Proposition 14.5.31 defines the 
function, 11: Equiv(X) Part(X), given by, 

U{R)=X/R={[x]R\xeX}, 

where R is any equivalence relation on X. We also write Ur instead of Il{R). 

There is also a function, TZ: Part(X) Equiv(X), that assigns an equivalence relation 
to a partition a shown by the next proposition. 

Proposition 4.5.5 For any partition, U = {Xi}i^j, on a set, X, the relation, 7^.(11), defined 
by 

xn{U)y iff (3«G/)(x,yGX,), 
is an equivalence relation whose equivalence classes are exactly the blocks Xi. 

Proof . We leave this easy proof as an exercise to the reader. □ 

Putting Propositions 14. 5. 3l and 14331 together we obtain the useful fact there is a bijection 
between Equiv(X) and Part(X). Therefore, in principle, it is a matter of taste whether we 
prefer to work with equivalence relations or partitions. In computer science, it is often 
preferable to work with partitions, but not always. 

Proposition 4.5.6 Given any set, X , the functions 11: Equiv(X) — > Part(X) and 
TZ: Part(X) —>■ Equiv(X) are mutual inverses, that is, 

7^ o n = id and H o 7^ = id. 

Consequently, there is a bijection between the set, Equiv(X), of equivalence relations on X 
and the set, Part(X), of partitions on X. 
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Proof. This is a routine verication left to the reader. □ 

Now, if / : X — >^ F is a surjective function, we have the equivalence relation, =j, defined 

by 

x=fy iff f{x) = f{y). 

It is clear that the equivalence class of any x G X is the inverse image, of 
f{x) G Y. Therefore, there is a bijection between X/ =f and Y. Thus, we can identify / 
and the projection, vr, from X onto X/ =/. If / is not surjective, note that / is surjective 
onto f{X) and so, we see that / can be written as the composition 

/ = Z O 77, 

where vr: X — /(X) is the canonical projection and i: /(X) ^ y is the inclusion function 
mapping /(X) into Y (i.e., i{y) = y, for every y G /(X)). 

Given a set, X, the inclusion ordering on X x X defines an ordering on binary relations 
on X, namely, 

R<S iff (Va;, y G X){xRy xSy). 

When R < S, we say that R refines S. If R and 5* are equivalence relations and R < S, 
we observe that every equivalence class of R is contained in some equivalence class of S. 
Actually, in view of Proposition I4.5.3[ we see that every equivalence class of S is the union 
of equivalence classes of R. We also note that idx is the least equivalence relation on X 
and X X X is the largest equivalence relation on X. This suggests the following question: 
Is Equiv(X) a lattice under refinement? 

The answer is yes. It is easy to see that the meet of two equivalence relations is Rd S, 
their intersection. But beware, their join is not RU S, because in general, RU S is not 
transitive. However, there is a least equivalence relation containing R and 5", and this is the 
join of R and 5*. This leads us to look at various closure properties of relations. 

4.6 Transitive Closure, Reflexive and Transitive Clo- 
sure, Smallest Equivalence Relation 

Let R be any relation on a set X. Note that R is reflexive iff idx ^ R- Consequently, the 
smallest reflexive relation containing R is idx U R. This relation is called the reflexive closure 
ofR. 

Note that R is transitive iS R o R (1 R. This suggests a way of making the smallest 
transitive relation containing R (if R is not already transitive). Define i?" by induction as 
follows: 

= idx 
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Definition 4.6.1 Given any relation, R, on a set, X, the transitive closure of R is the 
relation, R~^, given by 

R+ =\J i?". 

n>l 

The reflexive and transitive closure of R is the relation, R*, given by 

R* = \Jr^ = idx U R+. 

n>0 

The proof of the following proposition is left an an easy exercise. 

Proposition 4.6.2 Given any relation, R, on a set, X , the relation R^ is the smallest tran- 
sitive relation containing R and R* is the smallest reflexive and transtive relation containing 
R. 

If R is reflexive, then it is easy to see that R R"^ and so, i?^ C R'^^^ for all A; > 0. From 
this, we can show that if X is a finite set, then there is a smallest k so that R'' = R^^^. In 
this case, R^ is the reflexive and transitive closure of R. If X has n elements it can be shown 
that k < n — 1. 

Note that a relation, R, is symmetric iff R~^ = R. As a consequence, R U R~^ is the 
smallest symmetric relation containing R. This relation is called the symmetric closure of 
R. Finally, given a relation, R, what is the smallest equivalence relation containing Rl The 
answer is given by 

Proposition 4.6.3 For any relation, R, on a set, X , the relation 

{RuR-^y 

is the smalest equivalence relation containing R. 

4.7 Distributive Lattices, Boolean Algebras, Heyting 
Algebras 

If we go back to one of our favorite examples of a lattice, namely, the power set, 2^, of some 
set, X, we observe that it is more than a lattice. For example, if we look at Figure S31 we 
can check that the two identities Dl and D2 stated in the next definition hold. 

Definition 4.7.1 We say that a lattice, X, is a distributive lattice if (Dl) and (D2) hold: 



Dl a A (6 V c) = (a A 6) V (a A c) 
D2 aV (6Ac) = (a V6) A (a Vc). 



146 



CHAPTER 4. PARTIAL ORDERS AND EQUIVALENCE RELATIONS 



Remcirk: Not every lattice is distributive but many lattices of interest are distributive. 

It is a bit surprising that in a lattice, (Dl) and (D2) are actually equivalent, as we now 
show. Suppose (Dl) holds, then 

(a V fo) A (a V c) = ((a V 6) A a) V ((a V 6) A c) (Dl) 

= a V ((a V fe) A c) (L4) 

= a V ((cA (a V6)) (LI) 

= aV((cAa) V(cA6)) (Dl) 

= a V ((a Ac) V (6 Ac)) (LI) 

= (a V (a Ac)) V (6 Ac) (L2) 

= ((a A c) V a) V (6 A c) (LI) 

= a V (6 A c) (L4) 

which is (D2). Dually, (D2) implies (Dl). 

The reader should prove that every totally ordered poset is a distributive lattice. The 
lattice N+ under the divisibility ordering also turns out to be a distributive lattice. 

Another useful fact about distributivity which is worth noting is that in any lattice 

aA(6Vc) > (aA6) V(aAc). 

This is because in any lattice, a A {b\/ c) > a Ab and a A (6 V c) > a Ac. Therefore, in order 
to establish associativity in a lattice it suffices to show that 

aA(6Vc) < (aA6) V(aAc). 

Another important property of distributive lattices is the following: 

Proposition 4.7.2 In a distributive lattice, X, if z A x = z Ay and z \J x = z \J y, then 
X = y (for all x,y, z E X ). 

Proof . We have 

X = {xy z) Ax (L4) 

= xA{z\/x) (LI) 
= X A {z\/ y) 

= {x Az)\/ {x Ay) (Dl) 

= {z Ax)\/ {x Ay) (LI) 
= {z Ay)\/ {x Ay) 

^{y Az)y {y Ax) (LI) 

^y A{zy x) (Dl) 
^yA{zyy) 

^{yVz)Ay (LI) 

= y, (L4) 
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that is, X = y, as claimed. □ 

The power set lattice has yet some additional properties having to do with complemen- 
tation. First, the power lattice 2^ has a least element = and a greatest element, 1 = X. 
If a lattice, X, has a least element, 0, and a greatest element, 1, the following properties are 
clear: For all a G X, we have 

aAO=0 aVO=a 
a Al = a aVl = l. 

More importantly, for any subset, A C X, we have the complement. A, of A in X, which 
satisfies the identities: 

AUA = X, AnA = ^. 

Moreover, we know that the de Morgan identities hold. The generalization of these properties 
leads to what is called a complemented lattice. 

Definition 4.7.3 Let X be a lattice and assume that X has a least element, 0, and a 
greatest element, 1 (we say that X is a bounded lattice). For any a G X, a complement of a 
is any element, 6 G X, so that 

a V 6 = 1 and a A 6 = 0. 
If every element of X has a complement, we say that X is a complemented lattice. 

Remarks: 

1. When = 1, the lattice X collapses to the degenerate lattice consisting of a single 
element. As this lattice is of little interest, from now on, we will always assume that 
0^1. 

2. In a complemented lattice, complements are generally not unique. However, as the 
next proposition shows, this is the case for distributive lattices. 

Proposition 4.7.4 Let X be a lattice with least element and greatest element 1. If X is 
distributive, then complements are unique if they exist. Moreover, if b is the complement of 
a, then a is the complement ofb. 

Proof . If a has two complements, bi and 62? then a A 61 = 0, a A 62 = 0, a V 61 = 1, and 
a V 62 = 1- By commutativity, if follows that 6iAa = 62Aa = and 6iVa = 62Va = l. By 
Proposition I4.7.2[ we deduce that bi = 62, that is, a has a unique complement. 

By commutativity, the equations 



a V 6 = 1 and a Ab = 
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are equivalent to the equations 

6 V a = 1 and 6 A a = 0, 

which shows that a is indeed a complement of b. By uniqueness, a is the complement of b. 
□ 

In view of Proposition 14.7.41 if X is a complemented distributive lattice, we denote the 
complement of any element, a G X, by a. We have the identities 

a V a = 1 
a A a = 
a = a. 

We also have the following proposition about the de Morgan laws. 

Proposition 4.7.5 Let X be a lattice with least element and greatest element 1. If X is 
distributive and complemented, then the de Morgan laws hold: 



aV b = aAb 



aAb = ay b. 

Proof. We prove that 

a V 6 = a A 6, 

leaving the dual identity as an easy exercise. Using the uniqueness of complements, it is 
enough to check that aAb works, i.e., satisfies the conditions of Definition I4.7.3[ For the 
first condition, we have 

(aV6)V(aA6) = ((a V 6) V a) A ((a V 6) V 6) 

= (a V (6 V a)) A (a V (6 V 6)) 

= (a V (a V 6)) A (a V 1) 

= ((aVa)V6)Al 

= (1V6)A1 

= 1 A 1 = 1. 

For the second condition, we have 

(a V 6) A (a A 5) = (a A (a A 6)) V (6 A (a A 6)) 

= ((a A a) A 6) V (6 A (6 A a)) 

= (0 A 6) V ((6 A 6) A a) 

= V (0 A a) 

= 0V0 = 0. 



□ 

All this leads to the definition of a boolean lattice 
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Definition 4.7.6 A Boolean lattice is a lattice with a least element, 0, a greatest element, 
1, and which is distributive and complemented. 

Of course, every power set is a boolean lattice, but there are boolean lattices that are not 
power sets. Putting together what we have done, we see that a boolean lattice is a set, X, 
with two special elements, 0, 1, and three operations. A, V and a i— >■ a satisfying the axioms 
stated in 

Proposition 4.7.7 If X is a boolean lattice, then the following equations hold for all 
a,b,c E X : 



LI 


ay h = by a. 




a Ab = b A a 


L2 


(a V 6) V c = 


ay (bye), 


{a Ab) A c = a A {b A c) 


L3 


a V a = a. 




a A a = a 


LA 


(a V 6) A a = 


a. 


{a Ab) y a = a 


D1-D2 


a A (6 V c) = 


(a A 6) V (a A c). 


a V (6 A c) = (a V 6) A (a V c 


LE 


a V = a. 




a AO = 


GE 


a V 1 = 1, 




a Al = a 


C 


a V a = 1, 




a Aa = 


I 


a = a 






dM 


ay b = a Ab. 




a Ab = ay b. 



Conversely, if X is a set together with two special elements, 0, 1, and three operations. A, V 
and a I— > a satisfying the axioms above, then it is a boolean lattice under the ordering given 
by a < b iff ay b = b. 

In view of Proposition 14.7.71 we make the definition: 

Definition 4.7.8 A set, X, together with two special elements, 0, 1, and three operations, 
A, V and a a satisfying the axioms of Proposition 14.7.71 is called a Boolean algebra. 

Proposition 14.7.71 shows that the notions of a Boolean lattice and of a Boolean algebra 
are equivalent. The first one is order-theoretic and the second one is algebraic. 

Remarks: 

1. As the name indicates. Boolean algebras were invented by G. Boole (1854). One of the 
first comprehensive accounts is due to E. Schroder (1890-1895). 

2. The axioms for Boolean algebras given in Proposition 14. 7. 71 are not independent. There 
is a set of independent axioms known as the Huntington axioms (1933). 
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Let p be any integer with p > 2. Under the division ordering, it turns out that the set, 
Div(p), of divisors of p is a distributive lattice. In general not every integer, k G Div(p), has 
a complement but when it does, k = p/k. It can be shown that Div(p) is a Boolean algebra 
iff p is not divisible by any square integer (an integer of the form m^, with m > 1). 

Classical logic is also a rich source of Boolean algebras. Indeed, it is easy to show that 
logical equivalence is an equivalence relation and, as Homework problems, you have shown 
(with great pain) that all the axioms of Proposition 14.7.71 are provable equivalences (where 
V is disjunction, A is conjunction, P = -iP, i.e., negation, = _L and 1 = T). Furthermore, 
again, as a Homework problem, you have shown that logical equivalence is compatible with 
V, A, -1 in the following sense: If Pi = Qi and P2 = Q2, then 



i.e., iff P = Q is provable from T (as explained in Section [T7r|) . Clearly, =7- is an equiva- 
lence relation on propositions and so, we can define the operations V, A and - on the set of 
equivalence classes, B^, of propositions as follows: 



We also let = [±] and 1 = [T]. Then, we get the Boolean algebra, B^, called the 
Lindenhaum algebra of T. 

It also turns out that Boolean algebras are just what's needed to give truth- value seman- 
tics to classical logic. Let B be any Boolean algebra. A truth assignment is any function, 
u, from the set PS = {Pi, P2, ■ ■ ■ } of propositional symbols to B. Then, we can evaluate 
recursively the truth value, -Peff], in B of any proposition, P, with respect to the truth 
assigment, f , as follows: 



{Pi V P2) 
{Pi A P2) 
-Pi 



{Qi V Q2) 

{Qi A Q2) 

^Qi. 



Consequently, for any set, T, of propositions we can define the relation, =7^, by 



P=tQ iff Th P = Q 



[P] V [Q] 

[p]A[g] 
[p] 



[pvg] 

[PAQ] 

bP]- 




V 



V 



V 



v{P) 







1 



{pyQ)B 

{PAQ)b 
{-P)b 



V 



V 



V 



PbN VPbH 
Pb[v]APb[v] 
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In the equations above, on the right hand side, V and A are the lattice operations of the 
Boolean algebra, B. We say that a proposition, P, is valid in the Boolean algebra B (or 
B -valid) if Pb[v] = 1 for all truth assigments, v. We say that P is (classially) valid if P is 
5-valid in all Boolean algebras, B. It can be shown that every provable proposition is valid. 
This property is called soundness . Conversely, if P is valid, then it is provable. This second 
property is called completeness. Actually completeness holds in a much stronger sense: If a 
proposition is valid in the two element Boolean algebra, {0, 1}, then it is provable! 

One might wonder if there are certain kinds of algebras similar to Boolean algebras well 
suited for intuitionistic logic. The answer is yes: Such algebras are called Heyting algebras. 

In our study of intuitionistic logic, we learned that negation is not a primary connective 
but instead it is defined in terms of implication by -iP = P =>^-. This suggests adding to 
the two lattice operations V and A a new operation, — that will behave like The trick 
is, what kind of axioms should we require on — to "capture" the properties of intuitionistic 
logic? Now, if X is a lattice with and 1, given any two elements, a, 6 G X, experience 
shows that a b should be the largest element, c, such that c A a < b. This leads to 

Definition 4.7.9 A lattice, X, with and 1 is a Heyting lattice iff it has a third binary 
operation, such that 

c A a < 6 iff c< (a ^ b) 
for all a,b,c & X. We define the negation (or pseudo- complement) of a as a = (a ^ 0). 



At first glance, it is not clear that a Heyting lattice is distributive but in fact, it is. The 
following proposition (stated without proof) gives an algebraic characterization of Heyting 
lattices which is useful to prove various properties of Heyting lattices. 

Proposition 4.7.10 Let X be a lattice with and 1 and with a binary operation, — Then, 
X is a Heyting lattice iff the following equations hold for all a,b,c & X : 





a 


■ a 


= 1 


a A 


{a — > 


b) 


= a Ab 


bA 


(a —>■ 


b) 


= b 


a — : 


> (6 A 


c) 


= (a — s> 



A lattice with and 1 and with a binary operation, — >, satisfying the equations of 
Proposition 14.7.101 is called a Heyting algebra. So, we see that Proposition 14.7.101 shows 
that the notions of Heyting lattice and Heyting algebra are equivalent (this is analogous to 
Boolean lattices and Boolean algebras). 

The reader will notice that these axioms are propositions that were shown to be provable 
intuitionistically in Homework Problems! The proof of Proposition 14.7.101 is not really diffi- 
cult but it is a bit tedious so we will omit it. Let us simply show that the fourth equation 
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implies that for any fixed a & X, the map 6 i— > (a — > 6) is monotonic. So, assume b < c, i.e., 
bAc — b. Then, we get 

a — >• 6 = a — >• (6 A c) = (a — >• 6) A (a — >• c), 

which means that (a — > 6) < (a — > c), as claimed. 

The following theorem shows that every Heyting algebra is distributive, as we claimed 
earher. This theorem also shows "how close" to a Boolean algebra a Heyting algebra is. 

Theorem 4.7.11 (a) Every Heyting algebra is distributive. 

(h) A Heyting algebra, X , is a boolean algebra iff a = a for all a E X . 

Proof . (a) From a previous remark, to show distributivity, it is enough to show the inequality 

a A (6 V c) < (a A 6) V {a Ac). 
Observe that from the property characterizing we have 

b <a ^ {a Ab) iff b Aa < a Ab 

which holds, by commutativity of A. Thus, b < a ^ {a Ab) and similarly, c < a ^ (a A c). 

Recall that for any fixed a, the map x {a ^ x) is monotonic. Since aAb < (aA6) V(aAc) 
and a A c < (a A 6) V (a A c), we get 

a ^ (a A 6) < a ^ {{a Ab) V {a A c)) and {a Ac) < a ^ {{a Ab) V {a A c)). 

These two inequalities imply (a — (a A 6)) V (a — (a A c)) < a — >■ ((a A 6) V (a A c)), and 
since we also have & < a — > (a A 6) and c < a — > (a A c), we deduce that 

6 V c < a ^ ((a A 6) V {a Ac)), 

which, using the fact that (6 V c) A a = a A (6 V c), means that 

a A (6 V c) < (a A 6) V {a Ac), 

as desired. 

(b) We leave this part as an exercise. The trick is to see that the de Morgan laws hold 
and to apply one of them to a A a = 0. □ 

Reiricirks: 

1. Heyting algebras were invented by A. Heyting in 1930. Heyting algebras are sometimes 
known as "Brouwerian lattices" . 

2. Every Boolean algebra is automatically a Heyting algebra: Set a — > 6 = a V 6. 
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3. It can be shown that every finite distributive lattice is a Heyting algebra. 

We conclude this brief exposition of Heyting algebras by explaining how they provide 
a truth semantics for intuitionistic logic analogous to the thuth semantics that Boolean 
algebras provide for classical logic. 

As in the classical case, it is easy to show that intuitionistic logical equivalence is an 
equivalence relation and you have shown (with great pain) that all the axioms of Heyting 
algebras are intuitionistically provable equivalences (where V is disjunction, A is conjunction, 
and —>■ is =^). Furthermore, you have also shown that intuitionistic logical equivalence is 
compatible with V, A, =^ in the following sense: If Pi = Qi and P2 = Q2, then 



i.e., iff P = Q is provable intuitionistically from T (as explained in Section [L7|) . Clearly, =t 
is an equivalence relation on propositions and we can define the operations V, A and on 
the set of equivalence classes, H^, of propositions as follows: 



We also let = [±] and 1 = [T]. Then, we get the Heyting algebra, H^, called the 
Lindenhaum algebra of T, as in the classical case. 

Now, let H be any Heyting algebra. By analogy with the case of Boolean algebras, a truth 
assignment is any function, v, from the set PS = {Pi, P2, ■ ■ ■ } of propositional symbols to 
H. Then, we can evaluate recursively the truth value, Ph[v\i in H of any proposition, P, 
with respect to the truth assigment, v, as follows: 



(Pi V P2) 
[Pi A P2) 
{Pi P2) 



(Qi A Q2) 



Consequently, for any set, T, of propositions we can define the relation, =t 



by 



P=tQ iff ThP = Q, 



[P] V [Q] 
[P] A [Q] 
[P] - [Q] 



[pvg] 

[PAQ] 

[P^Q] 




v{P) 







1 



ipyQ)H[v. 

iPAQ)H[v, 

{P^Q)h[v 

hP)H[v: 



PH[v]yPH[v] 

Ph[v]APh[v] 



{Ph[v] ^ Ph[v]) 
(PhIv] 0). 
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In the equations above, on the right hand side, V, A and — > are the operations of the Heyting 
algebra, H. We say that a proposition, P, is valid in the Heyting algebra H (or H -valid) if 
PH-[f] = 1 for all truth assigments, v. We say that P is HA-valid (or intuitionistically valid) 
if P is H-YdXid in all Heyting algebras, H . As in the classical case, it can be shown that 
every intuitionistically provable proposition is HA-valid. This property is called soundness. 
Conversely, if P is HA-vahd, then it is intuitionistically provable. This second property is 
called completeness . A stronger completeness result actually holds: If a proposition is in- 
valid in all finite Heyting algebras, H ^ then it is intuitionistically provable. As a consequence, 
if a proposition is not provable intuitionistically, then it can be falsified in some finite Heyting 
algebra. 

Remcirk: If X is any set, a topology on X is a family, O, of subsets of X satisfying the 
following conditions: 

(1) e O and X eO] 

(2) For every family (even infinite), {Ui)i(zj, of sets Ui G O, we have IJ^^^ Ui G O. 

(3) For every finite family, (t/j)i<j<„, of sets Ui G O, we have flKKn ^ ^■ 

Every subset in O is called an open subset of X (in the topology O) . The pair, {X, O) , is 
called a topological space. Given any subset, ^4, of X, the union of all open subsets contained 

o 

in A is the largest open subset of A and is denoted A. 

Given a topological space, (X, O) , we claim that O with the inclusion ordering is a 
Heyting algebra with = 0; 1 — X; V = U (union); A = fl (intersection); and with 

o 

{u v)=\x-u)yjv . 

(Here, X — U \s the complement of U in X.) In this Heyting algebra, we have 

o 

u^T^. 

Since X — U is usually not open, wc generally have U ^ U . Therefore, we see that topology 
yields another supply of Heyting algebras. 



Chapter 5 

Graphs, Basic Notions 



5.1 Why Graphs? Some Motivations 

Graphs are mathematical structures that have many apphcations to computer science, elec- 
trical engineering and more widely to engineering as a whole, but also to sciences such as 
biology, linguistics, and sociology, among others. For example, relations among objects can 
usually be encoded by graphs. Whenever a system has a notion of state and state transition 
function, graph methods may be applicable. Certain problems are naturally modeled by 
undirected graphs whereas others require directed graphs. Let us give a concrete example. 

Suppose a city decides to create a public-transportation system. It would be desirable if 
this system allowed transportation between certain locations considered important. Now, if 
this system consists of buses, the traffic will probably get worse so the city engineers decide 
that the traffic will be improved by making certain streets one-way streets. The problem then 
is, given a map of the city consisting of the important locations and of the two-way streets 
linking them, find an orientation of the streets so that it is still possible to travel between any 
two locations. The problem requires finding a directed graph, given an undirected graph. 
Figure 15.11 shows the undirected graph corresponding to the city map and Figure 15.21 shows 
a proposed choice of one-way streets. Did the engineers do a good job or are there locations 
such that it is impossible to travel from one to the other while respecting the one-way signs? 

The answer to this puzzle will be revealed in Section [5^31 

There is a peculiar aspect of graph theory having to do with its terminology. Indeed, 
unlike most branches of mathematics, it appears that the terminology of graph theory is 
not standardized, yet. This can be quite confusing to the beginner who has to struggle with 
many different and often inconsistent terms denoting the same concept, one of the worse 
being the notion of a path. Our attitude has been to use terms that we feel are as simple as 
possible. As a result, we have not followed a single book. Among the many books on graph 
theory, we have been inspired by the classic texts, Harary [28], Berge [3] and BoUobas [7j. 
This chapter on graphs is heavily inspired by Sakarovitch [37], because we find Sakarovitch's 
book extremely clear and because it has more emphasis on applications than the previous 
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Figure 5.2: A choice of one-way streets 



5.2. DIRECTED GRAPHS 
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two. Another more recent (and more advanced) text which is also excellent is Diestel |13] . 

Many books begin by discussing undirected graphs and introduce directed graph only 
later on. We disagree with this approach. Indeed, we feel that the notion of a directed 
graph is more fundamental than the notion of an undirected graph. For one thing, a unique 
undirected graph is obtained from a directed graph by forgetting the direction of the arcs, 
whereas there are many ways of orienting an undirected graph. Also, in general, we believe 
that most definitions about directed graphs are cleaner than the corresponding ones for 
undirected graphs (for instance, we claim that the definition of a directed graph is simpler 
than the definition of an undirected graph, and similarly for paths). Thus, we begin with 
directed graphs. 

5.2 Directed Graphs 

Informally, a directed graph consists of a set of nodes together with a set of oriented arcs 
(also called edges) between these nodes. Every arc has a single source (or initial point) and 
a single target (or endpoint), both of which are nodes. There are various ways of formalizing 
what a directed graph is and some decisions must be made. Two issues must be confronted: 

1. Do we allow "loops," that is, arcs whose source and target are identical? 

2. Do we allow "parallel arcs," that is distinct arcs having the same source and target? 

Since every binary relation on a set can be represented as a directed graph with loops, our 
definition allows loops. Since the directed graphs used in automata theory must accomodate 
parallel arcs (usually labeled with different symbols), our definition also allows parallel arcs. 
Thus, we choose a more inclusive definition in order to accomodate as many applications 
as possible, even though some authors place restrictions on the definition of a graph, for 
example, forbidding loops and parallel arcs (we will call such graphs, simple graphs). Before 
giving a formal definition, let us say that graphs are usually depicted by drawings (graphs!) 
where the nodes are represented by circles containing the node name and oriented line 
segments labeled with their arc name (see Figure [375]) . 

Definition 5.2.1 A directed graph (or digraph) is a quadruple, G = {V,E,s,t), where V is 
a set of nodes or vertices, E is a set of arcs or edges and s,t: E ^ V are two functions, s 
being called the source function and t the target function. Given an edge e E E, we also call 
s(e) the origin or source of e, and t{e) the endpoint or target of e. 

If the context makes it clear that we are dealing only with directed graphs, we usually 
say simply "graph" instead of "directed graph". A directed graph, G = {V,E,s,t), is finite 
iff both V and E are finite. In this case, \V\, the number of nodes of G is called the order 
of G. 

Example: Let Gi be the directed graph defined such that 
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Figure 5.3: A directed graph, Gi 



E = {ei, 62, 63, 6^, 65, Ce, 67, Cg, Cg}, 

V = {fi,t;2,^^3,^^4,^^5,^^6}, and 



s(ei) 


= Vi, 3(62) 


= V2,S{63) 


= V3 


, 3(64) 


= V4, 


5(65) 


= V2,s{6e) 


= V5,S{67) 


= V5 


,s{6s) 


= V5,3{69) = Ve 


t(ei) 


= V2,t{62) 


= V3,t{63) 


= v^. 


t{64) -- 


= V2, 


^(65) 


= V5,t{66) 


= V5,t{67) 


= Ve, 


ties) -- 


= Ve,t{69) = V4. 



The graph Gi is represented by the diagram shown in Figure 15.31 

It should be noted that there are many different ways of "drawing" a graph. Obviously, 
we would like as much as possible to avoid having too many intersecting arrows but this is 
not always possible if we insist in drawing a graph on a sheet of paper (on the plane). 

Definition 5.2.2 Given a directed graph, G, an edge, 6 E E, such that s(e) = t(e) is called 
a loop (or self-loop). Two edges, 6,6' E E are said to be parallel edges iff s(e) = s(e') and 
t(e) = t(e'). A directed graph is simple iff it has no parallel edges. 
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Figure 5.4: A directed graph, G2 



Remarks: 



1. The functions s, t need not be injective or surjective. Thus, we allow "isolated vertices" , 
that is, vertices that are not the source or the target of any edge. 

2. When G is simple, every edge, e G -E, is uniquely determined by the ordered pair of 
vertices, {u,v), such that u = s{e) and v = t(e). In this case, we may denote the edge 
e by (uv) (some books also use the notation uv). Also, a graph without parallel edges 
can be defined as a pair, (V, E), with E (1 V x V. In other words, a simple graph is 
equivalent to a binary relation on a set {E ^ V x V) . This definition is often the one 
used to define directed graphs. 

3. Given any edge, e & E, the nodes s(e) and t{e) are often called the boundaries of e 
and the expression t(e) — s{e) is called the boundary of e. 

4. Given a graph, G = (V, E, s, t), we may also write V{G) for V and E{G) for E. Some- 
times, we even drop s and t and write simply G = {V, E) instead of G = {V, E, s, t). 

5. Some authors define a simple graph to be a graph without loops and without parallel 
edges. 



Observe that the graph Gi has the loop ee and the two parallel edges 67 and e^- When 
we draw pictures of graphs, we often omit the edge names (sometimes even the node names) 
as illustrated in Figure 15. 4[ 

Definition 5.2.3 Given a directed graph, G, for any edge e G -E, if m = s(e) and v = t{e), 
we say that 



(i) The nodes u and v are adjacent 
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(ii) The nodes u and v are incident to the arc e 

(iii) The arc e is incident to the nodes u and v 

(iv) Two edges, e, e' G -E are adjacent if they are incident to some common node (that is, 
either s(e) = s(e') or t(e) = t(e') or t(e) = s(e') or s(e) = t{e')). 

For any node, u , set 

(a) (i^(^) = l{^ ^ I '^(^) = ^}l ) the owter half-degree or outdegree of u 

(b) (i(j('u) = \{e E E \ t{e) = u}\ , the mner half-degree or indegree of u 

(c) dciu) = d^{u) + ' degree of u. 

A graph is regular iff every node has the same degree. 

Note that (respectively dQ{u)) counts the number of arcs "coming out from u" , that 
is, whose source is u (resp. counts the number of arcs "coming into u" , that is, whose target 
is u). For example, in the graph of Figure EH dQ^{vi) = 2, (i^^(t>i) = 1, (i^^(t>5) = 2, 
^^^(fs) = 4, dQ^{vs) = 2, dQ^{v3) = 2. Neither Gi nor G2 are regular graphs. 

The first result of graph theory is the following simple but very useful proposition: 
Proposition 5.2.4 For any finite graph, G = (y,E,s,t), we have 

^d+(n) = J^dci^)- 

Proof . Every arc, e G E, has a single source and a single target and each side of the above 
equations simply counts the number of edges in the graph. □ 

Corollary 5.2.5 For any finite graph, G = {V,E,s,t), we have 

J2dG{u) = 2\E\, 

that is, the sum of the degrees of all the nodes is equal to twice the number of edges. 

Corollary 5.2.6 For any finite graph, G = {V,E,s,t), there is an even number of nodes 
with an odd degree. 

The notion of homomorphism and isomorphism of graphs is fundamental. 
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Definition 5.2.7 Given two directed graphs, Gi = (Vi, Ei, Si,ti) and G2 = (V2, -E2, ^2), 
a homomorphism (or morphism), f : Gi ^ G2, from Gi to G2 is a pair, / = {f^,f^), with 
Vi V2 and f^: Ei ^ E2 preserving incidence, that is, for every edge, e G -Ei, we have 

52(r(e)) = r(.i(e)) and ^^(/^(e)) = r(ti(e)). 

These conditions can also be expressed by saying that the following two diagrams commute: 



El 



E. 



V2 



El 



E2 

t2 
V2. 



Given three graphs, Gi, 6*2,^3 and two homomorphisms, /: Gi — > 6*2 and g: G2 ^ G3, 
with / = {f", f^) and g = {g"", g^), it is easily checked that {g^of^, g^of^) is a homomorphism 
from Gi to G3. The homomorphism {g'" o (7^ o f^) is denoted go f. Also, for any graph, G, 
the map idg = (idy,id£;) is a homomorphism called the identity homomorphism. Then, a 
homomorphism, f: Gi G2, is an isomorphism iff there is a homomorphism, g: G2 —>■ Gi, 
such that 

gof = idG, and fog = idG2- 

In this case, g is unique and it is called the inverse of / and denoted /^^. If / = {f^, /^) is 
an isomorphism, we see immediately that and are bijections. Ghecking whether two 
finite graphs are isomorphic is not as easy as it looks. In fact, no general efficient algorithm 
for checking graph isomorphism is known at this time and determining the exact complexity 
of this problem is a major open question in computer science. For example, the graphs 
G3 and G4 shown in Figure 1531 are isomorphic. The bijection f" is given by f^{vi) = Wi, 
for i = 1, . . . , 6 and the reader will easily figure out the bijection on arcs. As we can see, 
isomorphic graphs can look quite different. 



5.3 Paths in Digraphs; Strongly Connected Compo- 
nents 

Many problems about graphs can be formulated as path existence problems. Given a directed 
graph, G, intuitively, a path from a node u to a node f is a way to travel from m in f by 
following edges of the graph that "link up correctly". Unfortunately, if we look up the 
definition of a path in two different graph theory books, we are almost guaranteed to find 
different and usually clashing definitions! This has to do with the fact that for some authors, 
a path may not use the same edge more than once and for others, a path may not pass through 
the same node more than once. Moreover, when parallel edges are present (i.e.. when a graph 
is not simple), a sequence of nodes does not define a path unambiguously!. 
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Figure 5.5: Two isomorphic graphs, and G4 



The terminology that we have chosen may not be standard, but it is used by a number 
of authors (some very distinguished, for example. Fields medalists!) and we believe that it 
is less taxing on one's memory (however, this point is probably the most debatable). 

Definition 5.3.1 Given any digraph, G = {V,E,s,t), and any two nodes, u,v E V , a. path 
from u to V is a triple, tt = (m, ei • • • e„, v), where n > 1 and ei • • • is a sequence of edges, 
Ci E E (i.e., a nonempty string in E*), such that 

s(ei) = u; t{en) = v; t{ei) = s{ei+i), 1 < i < n - 1. 

We call n the length of the path n and we write |7r| = n. When n = 0, we have the null path, 
{u, e, u), from uto u (recall, e denotes the empty string); the null path has length 0. Uu — v, 
then TT is called a closed path, else an open path. The path, tt = {u,ei - ■ ■ e^, v), determines 
the sequence of nodes, nodes(7r) = {uq, . . . ,Un), where Uq — u, Un — v and Ui — t{ei), for 
1 < i < n — 1. We also set nodes((?i, e, u)) = {u, u). A path, tt = {u,ei - ■ ■ e„, v), is simple 
iff Cj 7^ Cj for all i ^ j (i.e., no edge in the path is used twice). A path, vr, from u io v 
is elementary iff no vertex in nodes (tt) occurs twice, except possibly for m if tt is closed. 
Equivalently, if nodes(7r) = {uq, . . . , Un), then tt is elementary iff either 

1. Uj for all i,j with i ^ j and < i, j < n, or tt is closed {uq = Un) in which case 

2. Ui 7^ Uq and Ui 7^ m„ for all i with 1 < i < n — 1, and Ui ^ Uj for all i, j with i ^ j and 
1 < i, j < n — 1. 

The null path, {u, e, u), is considered simple and elementary. 
Remctrks: 

1. Other authors use the term walk for what we call a path. These authors also use 
the term trail for what we call a simple path and the term path for what we call an 
elementary path. 
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2. If a digraph is not simple, then even if a sequence of nodes is of the form nodes(7r) for 
some path, that sequence of nodes does not uniquely determine a path. For example, 
in the graph of Figure [5T3l the sequence {v2,V5,vq) corresponds to the two distinct 
paths (f2,e5e7,f6) and (^2, 6563, fe)- 

In the graph Gi from Figure 15.31 

(f2, 656769646568, Vq) 

is a path from V2 to Vq which is neither simple nor elementary, 

(t;2, 62636465, f5) 
is a simple path from V2 to which is not elementary and 

(f 2, 656769,^4), (f2, 65676964, f2) 

are elementary paths, the first one open and the second one closed. 

Recall the notion of subsequence of a sequence defined just before stating Theorem 12.9.81 
Then, if vr = (m, ci ■ ■ ■ 6„, v) is any path from m to v in a digraph, G, a subpath of vr is any 
path vr' = (m, e[ - ■ ■ e'^, v) such that e[, . . . , 6'^ is a subsequence of 61, . . . , 6„. The following 
simple proposition is actually very important: 

Proposition 5.3.2 Let G be any digraph, (a) For any two nodes, u,v, in G, every non-null 
path, IT, from u to v contains an elementary non-null subpath. 

(b) If \ V\ = n, then every open elementary path has length at most n — 1 and every closed 
elementary path has length at most n. 

Proof . (a) Let vr be any non-null path from m to f in G and let 

S* = {A; G N I A; = |7r'|, vr' is a non-null subpath of vr}. 

The set 5* C N is nonempty since |7r| G 5 and as N is well-ordered, S has a least element, 
say m > 1. We claim that any subpath of vr of length m is elementary. Consider any such 
path, say n' = {u, e[ - ■ ■ e'^, v), let 

nodes(7r') = {vq, ...,Vm), 
and assume that vr' is not elementary. There are two cases: 

(1) u ^ V. Then, some node occurs twice in nodes(7r'), say Vi = Vj, with i < j. Then, we 
can delete the path (fi,6^_,_^ ■ ■ ■ ,e'j,Vj) from vr' to obtain a non-null (because u v) 
subpath vr" of vr' from m to f with |7r"| = |7r'| — (j — i) and since i < j, we see that 
Ivr"! < Itt'I, contradicting the minimality of m. Therefore, vr' is a non-null elementary 
subpath of TT. 
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(2) u = V. In this case, either some node occurs twice in the sequence (fo, . . . ,fm.-i) or 
some node occurs twice in the sequence (f i, . . . , Vm)- In either case, as in (1), we can 
strictly shorten the path from vq to Vm-i or the path from Vi to Vm- Even though 
the resulting path may be the null path, as one of two two edges e'^ or remains 
from the original path tt', we get a non-null path from m to m strictly shorter than vr', 
contradicting the minimality of tt'. 

(b) As in (a), let vr' be an open elementary path from m to f and let 

nodes(7r') = (wq, ...,v^). 

If m > 77, = 1 1^1, as the above sequence has m + 1 > n nodes, by the Pigeonhole Principle, 
some node must occur twice, contradicting the fact that vr' is an open elementary path. If vr' 
is a non-null closed path and m > n + 1, then by the Pigeonhole Principle, either some node 
occurs twice in (fo, . . . , Vm) or some node occurs twice in (t>i, . . . , Vm+i)- In either case, this 
contradicts the fact that vr' is a non-null elementary path. □ 

Like strings, paths can be concatenated. 

Definition 5.3.3 Two paths, tt = (-u, ei ■ ■ • Cm, v) and vr' = e'^ ■ ■ ■ e^, v') in a digraph G 
can be concatenated iff w = m' in which case their concatenation, nn', is the path 

Tin' = {u,ei ■ • -e^e'i ■ ■■e'^,v'). 

We also let 

(m, e, u)7i = 71 = 7i{v, e, v). 

Concatenation of paths is obviously associative and observe that Itttt'I = |7r| + \7i'\. 

Definition 5.3.4 Let G = {V,E,s,t) be a digraph. We define the binary relation, Cg, on 
V as follows: For all u,v & V, 

uGgv iff there is a path from u to f and there is a path from v to u. 

When uGgV, we say that u and v are strongly connected. 

Oberve that the relation Gg is an equivalence relation. It is reflexive because we have 
the null path from u to u, symmetric by definition, and transitive because paths can be 
concatenated. The equivalence classes of the relation Cg are called the strongly connected 
components ofG (SCO's). A graph is strongly connected iff it has a single strongly connected 
component. 

For example, we see that the graph, Gi, of Figure [573] has two strongly connected com- 
ponents 

{Vi}, {v2,V3,V4,V5,Vfi}, 
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since there is a closed path 

(f4, 64626364656769, 1)4) . 

The graph G2 of Figure 15.41 is strongly connected. 

Let us give a simple algorithm for computing the strongly connected components of a 
graph since this is often the key to solving many problems. The algorithm works as follows: 
Given some vertex, u the algorithm computes the two sets, X~^{u) and X~{u), where 

X^(-u) = {t; G I there exists a path from u to v} 
X^{u) = {v ^ V \ there exists a path from v to u}. 

Then, it is clear that the connected component, C{u), or u, is given by C(m) = C'^{u)nX^ (u). 
For simplicity, we assume that X^{u),X~{u) and C{u) are represented by linear arrays. In 
order to make sure that the algorithm makes progress, we used a simple marking scheme. We 
use the variable total to count how many nodes are in X^{u) (or in X~{u)) and the variable 
marked to keep track of how many nodes in X'^('u) (or in X^{u)) have been processed so 
far. Whenever the algorithm considers some unprocessed node, the first thing it does is to 
increment marked by 1. Here is the algorithm in high-level form. 

function strcomp{G: graph; u: node): set 
begin 

X+('u)[l] := u; X^('u)[l] := u; total := 1; marked := 0; 
while marked < total do 

marked := marked + 1; v := X~^ {u)[marked]; 
for each e E E 

if (s(6) =v) A {t{e) i X+(m)) then 

total := total + 1; X^ {u)[total] := t{e) endif 
endfor 
endwhile; 

total := 1; marked := 0; 
while marked < total do 

marked := marked + 1; v := X^{u)[marked]; 
for each e E E 

if {t{e) =v)A {s{e) ^ X-{u)) then 

total := total + 1; X^{u)[total] := s{e) endif 
endfor 
endwhile; 

C{u) = X+(m) n X"(u); strcomp := C{u) 
end 

If we want to obtain all the strongly connected components (SCO's) of a finite graph, G, 
we proceed as follows: Set Vi = V, pick any node, Vi, in Vi and use the above algorithm 
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to compute the strongly connected component, Ci, of Vi. If Vi = Ci, stop. Otherwise, 
let V2 = Vi — Ci. Again, pick any node, V2 in V2 and determine the strongly connected 
component, C2, of V2. If V2 = C2, stop. Otherwise, let V3 = V2 — C2, pick t>3 in V3, and 
continue in the same manner as before. Ultimately, this process will stop and produce all 
the strongly connected components Ci, . . . ,Ck of G. 

It should be noted that the function strcomp and the simple algorithm that we just 
described are "naive" algorithms that are not particularly efficient. Their main advantage 
is their simplicity. There are more efficient algorithms, in particular, there is a beautiful 
algorithm for computing the SCO's due to Robert Tarjan. 

Going back to our city traffic problem from Section 15.11 if we compute the strongly 
connected components for the proposed solution shown in Figure 15.21 , we find three SOO's: 

{6,7,8,12,13,14}, {11}, {1,2,3,4,5,9,10,15,16,17,18,19}. 

Therefore, the city engineers did not do a good job! We will show after proving Proposition 
15.3.81 how to "fix" this faulty solution. 

Closed simple paths also play an important role. 

Definition 5.3.5 Let G = {V,E,s,t) be a digraph. A circuit is a closed simple path (i.e., 
no edge occurs twice) and an elementary circuit is an elementary closed path. The null path, 
(m, e, u), is an elementary circuit. 

Remark: A closed path is sometimes called a pseudo- circuit. In a pseudo-circuit, some 
edge may occur move than once. 

The significance of elementary circuits is revealed by the next proposition. 

Proposition 5.3.6 Let G be any digraph, (a) Every circuit, tt, in G is the concatenation 
of pairwise edge-disjoint elementary circuits. 

(b) A circuit is elementary iff it is a minimal circuit, that is, iff it does not contain any 
proper circuit. 

Proof. We proceed by induction on the length of tt. The proposition is trivially true if tt is 
the null path. Next, let tt = (m, ei ■ ■ ■ Cm, u) be any non-null circuit and let 

nodes(7r) = {vq, ...,Vm)- 

If TT is an elementary circuit, we are done. Otherwise, some node occurs twice in the sequence 
(fo, . . . , Vm-i) or in the sequence {vi, . . . , Vm)- Let us consider the first case, the second one 
being similar. Pick two occurrences of the same, node, say Vi = Vj, with i < j, such that 
j —i is minimal. Then, due to the minimality of j — i, no node occurs twice in {vi, . . . , f j-i) 
or {vi+i, . . . ,Vj), which shows that tti = (t>j, Cj+i ■ ■ ■ Cj, fj) is an elementary circuit. Now, we 
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can write tt = ttViTt", with \7i'\ < \7i\ and \7i"\ < \7i\. Thus, we can apply the induction 
hypothesis to both tt' and n", which shows that tt' and tt" are concatenations of elementary 
circuits. Then, tt itself is the concatenation of elementary circuits. All these elementary 
circuits are pairwise edge-disjoint since tt has no repeated edges. 

(b) This is clear by definition of an elementary circuit. □ 
Remarks: 

1. If M and V are two nodes that belong to a circuit, n, in G, (i.e., both u and v are incident 
to some edge in tt), then u and v are strongly connected. Indeed, u and v are connected 
by a portion of the circuit tt, and v and u are connected by the complementary portion 
of the circuit. 

2. If TT is a pseudo-circuit, the above proof shows that it is still possible to decompose tt 
into elementary circuits, but it may not be possible to write tt as the concatenation of 
pairwise edge-disjoint elementary circuits. 

Given a graph, G, we can form a new and simpler graph from G by connecting the 
strongly connected components of G as shown below. 

Definition 5.3.7 Let G = {V,E,s,t) be a digraph. The reduced graph, G, is the simple 
digraph whose set of nodes, V = V/Cg, is the set of strongly connected components of V 
and whose set of edges, E, is defined as follows: 

{u,v) e E iff (3e e E){s{e) e u and t{e) e v), 

where we denote the strongly connected component of u by u. 

That G is "simpler" than G is the object of the next proposition. 

Proposition 5.3.8 Let G be any digraph. The reduced graph, G, contains no circuits. 

Proof. Suppose that u and v are nodes of G and that u and v belong to two disjoint strongly 
connected components that belong to a circuit, tt, in G. Then, the circuit, tt, yields a closed 
sequence of edges ei, . . . , between strongly connected components and we can arrange the 
numbering so that these components are Cq, . . . ,Cn, with Cn — Co, with an edge between 
s(ej) G Ci and t(ej) G Cj+i for < i < rz, — 1, e„ an edge between between s(e„) G C„ 
and t(e„) G Cq, u = Cp and v = Cq, for some p < q. Now, we have t(ej) G Cj+i and 
s(ej+i) G Cj+i for < i < n — 1 and t(e„) G Cq and s(ei) G Cq and as each Cj is strongly 
connected, we have elementary paths from t{ei) to s(ei+i) and from t{en) to s(ei). Also, as 
u — Cp and v — Cq for some p < q, we have some elementary paths from rt to s(ep) and 
from t(eg_i) to f . By concatenating the appropriate paths, we get a circuit in G containing 
u and V, showing that u and v arc strongly connected, contradicting that u and v belong to 
two disjoint strongly connected components. □ 
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Figure 5.6: The reduced graph of the graph in Figure [52] 

Remark: Digraphs without circuits are called DAG's. Such graphs have many nice prop- 
erties. In particular, it is easy to see that any finite DAG has nodes with no incoming edges. 
Then, it is easy to see that finite DAG's are basically collections of trees with shared nodes. 

The reduced graph of the graph shown in Figure 15.21 is showed in Figure 15. 6[ where its 
SCO's are labeled A, B and C as shown below: 

A = {6,7,8,12,13,14}, B = {11}, C = {1, 2, 3, 4, 5, 9, 10, 15, 16, 17, 18, 19}. 

The locations in the component A are inaccessible. Observe that changing the direction 
of the street between 13 and 18 yields a solution, that is, a strongly connected graph. So, 
the engineers were not too far off after all! The solution to our traffic problem is shown in 
Figure 15. 7[ 




Figure 5.7: A good choice of one-way streets 



Before discussing undirected graphs, let us collect various definitions having to do with 
the notion of subgraph. 
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Definition 5.3.9 Given any two digraphs, G — {V,E,s,t) and G' — {V , E' , s' ,t'), we say 
that G' is a subgraph of G iH V C V, E' C E, s' is the retriction of s to E' and t' is the 
retriction of t to E'. If G' is a subgraph of G and V = V, we say that G' is a spanning 
subgraph of G. Given any subset, V, of V, the induced subgraph, G{V'), of G is the graph 
whose set of edges is 

Ev'^{eeE \ s{e) E V';t{e) E V'}. 

(Glearly, s' and f are the restrictions of s and t to Ey', respectively) Given any subset, 
E' C E, the graph G' — {V, E' , s' ,t'), where s' and t' are the restrictions of s and t to 
respectively, is called the partial graph ofG generated by E' . The graph, {V' , E'CWv'-i s', t'), is 
a partial subgraph ofG (here, s' and t' are the restrictions of s and i to E'nVv, respectively.) 

5.4 Undirected Graphs, Chains, Cycles, Connectivity 

The edges of a graph express relationships among its nodes. Sometimes, these relationships 
are not symmetric, in which case it is desirable to use directed arcs, as we have in the pre- 
vious sections. However, there is a class of problems where these relationships are naturally 
symmetric or where there is no a priori preferred orientation of the arcs. For example, if 
V is the population of individuals that were students at Pcnn between 1900 until now and 
if we are interested in the relation where two people A and B are related iff they had the 
same professor in some course, then this relation is clearly symmetric. As a consequence, 
if we want to find the set of individuals that are related to a given individual. A, it seems 
unnatural and, in fact, counter-productive, to model this relation using a directed graph. 

As another example suppose we want to investigate the vulnerabilty of an internet net- 
work under two kinds of attacks: (1) disabling a node; (2) cutting a link. Again, whether of 
not a link between two sites is oriented is irrelevant. What is important is that the two sites 
are either connected or disconnected. 

These examples suggest that we should consider an "unoriented" version of a graph. How 
should we proceed? 

One way to proceed is to still assume that we have a directed graph but to modify 
certain notions such as paths and circuits to account for the fact that such graphs are really 
"unoriented." In particular, we should redefine paths to allow edges to be traversed in the 
"wrong direction". Such an approach is possible but slightly akward and ultimately it is 
really better to define undirected graphs. However, to show that this approach is feasible, 
let us give a new definition of a path that corresponds to the notion of path in an undirected 
graph. 

Definition 5.4.1 Given any digraph, G = {V, E, s, t), and any two nodes, u,v E V . a chain 
(or walk) from u to v is a. sequence n = {uQ,ei,Ui,e2,U2, ■ ■ ■ ,Un-i,en,Un), where n > 1; 
Ui E V; Ci E E and 

Uo = u; Un = v and {s(ej), t(ej)} = {ui_i, Ui}, 1 < i < n. 
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We call n the length of the chain vr and we write |7r| = n. When n = 0, we have the null 
chain, {u, e, u), from -u to m, a chain of length 0. If m = v, then vr is called a closed chain, else 
an open chain. The chain, vr, determines the sequence of nodes, nodes(7r) = {uq, . . . ,Un), 
with nodes((M, e, u)) = {u, u). A chain, tt, is simple iff e, ^ Cj for all i ^ j (i.e., no edge in the 
chain is used twice). A chain, vr, from m to is elementary iff no vertex in nodes(7r) occurs 
twice, except possibly for m if vr is closed. The null chain, {u,e,u), is considered simple and 
elementary. 

The main difference between Definition 15.4.11 and Definition 15.3.11 is that Definition 15.4.11 
ignores the orientation: in a chain, an edge may be traversed backwards, from its endpoint 
back to its source. This implies that the reverse of a chain 

TT^ = {Un, e„, Un-l, , ■ ■ ■ ,U2, 62, Ml, 61,^0) 

is a chain from v = Un to u = uq. In general, this fails for paths. Note, as before, that if G 
is a simple graph, then a chain is more simply defined by a sequence of nodes 

(MO,Mi, . . . ,Un). 

For example, in the gaph G5 shown in Figure 15. 8[ we have the chains 

{vi, a, V2, d, f4, /, W5, e, V2, d, v^, g, v^), (vi, a, V2, d, v^, f, v^, e, V2, c, v^), (vi, a, V2, d, v^, g, V3) 

from vi to V3, The second chain is simple and the third is elementary. Note that none of 
these chains are paths. 

Chains are concatenated the same way as paths and the notion of subchain is analogous 
to the notion of subpath. The undirected version of Proposition 15.3.21 also holds. The proof 
is obtained by changing the word "path" to "chain" . 

Proposition 5.4.2 Let G be any digraph, (a) For any two nodes, u,v, in G, every non-null 
chain, n, from u to v contains an elementary non-null subchain. 

(b) If \V\ = n, then every open elementary chain has length at most n — 1 and every 
closed elementary chain has length at most n. 
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The undirected version of strong connectivity is the following: 

Definition 5.4.3 Let G = {V,E,s,t) be a digraph. We define the binary relation, Go, on 
V as follows: For all u,v E V, 

uGgv iff there is a chain from u io v. 

When uGgv, we say that u and v are connected. 

Oberve that the relation Go is an equivalence relation. It is reflexive because we have 
the null chain from u to u, symmetric because the reverse of a chain is also a chain and 
transitive because chains can be concatenated. The equivalence classes of the relation Gg 
are called the connected components of G (CC's). A graph is connected iff it has a single 
connected component. 

Observe that strong connectivity implies connectively but the converse is false. For 
example, the graph Gi of Figure 15.31 is connected but it is not strongly connected. The 
function strcomp and the method for computing the strongly connected components of a 
graph can easily be adapted to compute the connected components of a graph. 

The undirected version of a circuit is the following: 

Definition 5.4.4 Let G = {V, E, s, t) be a digraph. A cycle is a closed simple chain (i.e., no 
edge occurs twice) and an elementary cycle is an elementary closed chain. The null chain, 
(m, e, u), is an elementary cycle. 

Remark: A closed cycle is sometimes called a pseudo-cycle. The undirected version of 
Proposition 15.3.61 also holds. Again, the proof consist in changing the word "circuit" to 
"cycle". 

Proposition 5.4.5 Let G be any digraph, (a) Every cycle, tt, in G is the concatenation of 
pairwise edge-disjoint elementary cycles. 

(b) A cycle is elementary iff it is a minimal cycle, that is, iff it does not contain any 
proper cycle. 

The reader should now be convinced that it is actually possible to use the notion of a 
directed graph to model a large class of problems where the notion of orientation is irrelevant. 
However, this is somewhat unnatural and often inconvenient, so it is desirable to introduce 
the notion of an undirected graph as a "first-class" object. How should we do that? 

We could redefine the set of edges of an undirected graph to be of the form E^ U E^ , 
where E^ = E is the original set of edges of a digraph and with 

E- = {e- I e+ e E+, s{e-) = t(e+), t{e-) = s(e+)}. 
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each edge, e , being the "anti-edge" (opposite edge) of e"*". Such an approach is workable 
but experience shows that it not very satisfactory. 

The solution adopted by most people is to relax the condition that every edge, e & E, is 
assigned an ordered pair, {u,v), of nodes (with u = s(e) and v = t{e)) to the condition that 
every edge, e G is assigned a set, {u,v} of nodes (with u = v allowed). To this effect, 
let [V]'^ denote the subset of the power set consisting of all two-element subsets of V (the 
notation (^) is sometimes used instead of [V]"^) : 

[V]^ = {{u,v} \uy^v}. 

Definition 5.4.6 A graph is a triple, G = {V,E,st), where is a set of nodes or vertices, 
E is a set of arcs or edges and st: E ^ VUlV]"^ is a function that assigns a set of endpoints 
(or endnodes) to every edge. 

When we want to stress that we are dealing with an undirected graph as opposed to a di- 
graph, we use the locution undirected graph. When we draw an undirected graph we suppress 
the tip on the extremity of an arc. For example, the undirected graph, Gq, corresponding to 
the directed graph G5 is shown in Figure 15.91 

Definition 5.4.7 Given a graph, G, an edge, e & E, such that st(e) G is called a loop 
(or self-loop). Two edges, e,e' & E are said to be parallel edges iff st{e) = st{e'). A graph is 
simple iff it has no loops and no parallel edges. 

Remarks: 

1. The functions st need not be injective or surjective. 

2. When G is simple, every edge, e G -E, is uniquely determined by the set of vertices, 
{u, v}, such that {u, v} = st{e). In this case, we may denote the edge e by {u, v} (some 
books also use the notation (uv) or even uv). 
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3. Some authors call a graph with no loops but possibly parallel edges a multigraph and 
a graph with loops and parallel edges a pseudograph. We prefer to use the term graph 
for the most general concept. 

4. Given an undirected graph, G = {V,E,st), we can form directed graphs from G by 
assigning an arbitrary orientation to the edges of G. This means that we assign to 
every set, st{e) = {u,v}, where u ^ v, one of the two pairs {u,v) or {v,u) and define 
s and t such that s(e) = u and t{e) = v in the first case or such that s(e) = v and 
t{e) = M in the second case (when u = v, we have s(e) = t{e) = u)). 

5. When a graph is simple, the function st is often omitted and we simply write {V,E), 
with the understanding that E is a set of two-elements subsets of V. 

6. The concepts or adjacency and incidence transfer immediately to (undirected) graphs. 

It is clear that the Definition of chain, connectivity, and cycle (Definitions 15.4. H 15.4.31 
and 15.4.41) immediately apply to (undirected) graphs. However, only the notion of degree (or 
valency) of a node applies to undirected graph where it is given by 

dciu) = \{e e E \ u e st(e)}|. 

We can check immediately that Corollary 15.2.51 and Corollary 15.2.61 apply to undirected 
graphs. 

Remark: When it is clear that we are dealing with undirected graphs, we will sometimes 
allow ourselves some abuse of language. For example, we will occasionally use the term path 
instead of chain. 

The notion of homomorphism and isomorphism also makes sense for undirected graphs. 
In order to adapt Definition 15.2.71 observe that any function, g: Vi ^ V2, can be extended 
in a natural way to a function from Vi U [Vi]"^ to V2 U [V2]^, also denoted g, so that 

g{{u,v}) = {g{u),g{v)}, 

for all {u,v} G [Vi]'^. 

Definition 5.4.8 Given two graphs, Gi = {Vi, Ei, sti) and G2 = {V2, E2, st2), a homomor- 
phism (or morphism), f : Gi ^ G2, from Gi to G2 is a pair, / = (/^, f^), with f" : Vi V2 
and f^: El E2 preserving incidence, that is, for every edge, e G -Ei, we have 

st2{ne)) = r{sh{e)). 

These conditions can also be expressed by saying that the following diagram commute: 

fe 

El E2 

sti St2 
VlVj[Vif—^ V2U[V2f. 




Figure 5.10: A bridge in the graph Gj 



As for directed graphs, we can compose homomorphisms of undirected graphs and the 
definition of an isomorphism of undirected graphs is the same as the definition of an isomor- 
phism of digraphs. 

We are now going to investigate the properties of a very important subclass of graphs, 
trees. 

5.5 Trees and Arborescences 

In this section, until further notice, we will be dealing with undirected graphs. Given a 
graph, G, edges having the property that their deletion increases the number of connected 
components of G play an important role and we would like to characterize such edges. 

Definition 5.5.1 Given any graph, G = {V,E,st), any edge, e E E, whose deletion in- 
creases the number of connected components of G (i.e., (V, E — {e}, st \ {E — {e})) has more 
connected components than G) is called a bridge. 

For example, the edge {V4V5) in the graph shown in Figure [5TT0] is a bridge. 

Proposition 5.5.2 Given any graph, G = {V,E,st), adjunction of a new edge, e, between 
u and V (this means that st is extended to stg, with ste{e) = {u,v}) to G has the following 
effect: 

1. Either the number of components of G decreases by 1, in wich case the edge e does not 
belong to any cycle of G' = {V,E U {e}, st^), or 

2. The number of components of G is unchanged, in wich case the edge e belongs to some 
cycle of G' = {V, E U {e}, ste). 

Proof . Two mutually exclusive cases are possible: 
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(a) The endpoints u and v (of e) belong to two disjoint connected components of G. In 
G' , these components are merged. The edge e can't belong to a cycle of G' because the 
chain obtained by deleting e from this cycle would connect u and f in G, a contradiction. 

(b) The endpoints u and v (of e) belong to the same connected component of G. Then, 
G' has the same connected components as G. Since u and v are connected, there is an 
elementary chain from utov (by Proposition 15. 4. 2| and by adding e to this elementary 
chain, we get a cycle of G' containing e. □ 

Corollary 5.5.3 Given any graph, G = {V, E, st), an edge, e E E, is a bridge iff it does not 
belong to any cycle ofG. 

Theorem 5.5.4 Let G be a finite graph and letm = \V\ > 1. The following properties hold: 
(i) If G is connected, then \E\ > m — 1. 
(a) If G has no cycle, then \E\ < m — 1. 

Proof. We can build the graph G progressively by adjoining edges one at a time starting 
from the graph (V, 0), which has m connected components. 

(i) Every time a new edge is added, the number of connected components decreases by 
at most 1. Therefore, it will take at least m — 1 steps to get a connected graph. 

(ii) If G has no cycle, then every spannning graph has no cycle. Therefore, at every step, 
we are in case (1) of Proposition 15.5.21 and the number of connected components decreases by 
exactly 1. As G has at least one connected component, the number of steps (i.e., of edges) 
is at most m — 1. □ 

In view of Theorem I5.5.4[ it makes sense to define the following kind of graphs: 

Definition 5.5.5 A tree is a graph that is connected and acyclic (i.e., has no cycles). A 
forest is a graph whose connected components are trees. 

The picture of a tree is shown in Figure 15. Hi 

Our next theorem gives several equivalent characterizations of a tree. 

Theorem 5.5.6 Let G be a finite graph with m = \V\ > 2 nodes. The following properties 
characterize trees: 

(1) G is connected and acyclic. 

(2) G is connected and minimal for this property (if we delete any edge of G, then the 
resulting graph is no longer connected). 

(3) G is connected and has m — 1 edges. 
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Figure 5.11: A Tree, Ti 



(4) G is acyclic and maximal for this property (if we add any edge to G, then the resulting 
graph is no longer acyclic). 

(5) G is acyclic and has m — 1 edges. 

(6) Any two nodes of G are joined by a unique chain. 
Proof . The implications 

(1)^(3), (5) 
(3) =^ (2) 
(5) =^ (4) 

all follow immediately from Theorem I5.5.4[ 

(4) =^ (3). If G was not connected, we could add an edge between to disjoint connected 
components without creating any cycle in G, contradicting the maximality of G with respect 
to acyclicity. By Theorem I5.5.4I as G is connected and acyclic, it must have m — 1 edges. 

(2) =^ (6). As G is connected, there is a chain joining any two nodes of G. If, for two 
nodes u and v, we had two distinct chains from u to v, deleting any edge from one of these 
two chains would not destroy the connectivity of G contradicting the fact that G is minimal 
with respect to connectivity. 

(6) =^ (1). If G had a cycle, then there would be at least two distinct chains joining 
two nodes in this cycle, a contradiction. 

The reader should then draw the directed graph of implications that we just established 
and check that this graph is strongly connected! Indeed, we have the cycle of implications 

(1) =^ (5) =^ (4) =^ (3) =^ (2) =^ (6) =^ (1). 

□ 



Remark: The equivalence of (1) and (6) holds for infinite graphs too. 
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Figure 5.12: A Spanning Tree 



Corollary 5.5.7 For any tree, G, adding a new edge, e, to G yields a graph, G' , with a 
unique cycle. 

Proof. Because G is a tree, all cycles of G' must contain e. If G' had two distinct cycles, 
there would be two distinct chains in G joining the endpoints of e, contradicting property 
(6) of Theorem 15.5.61 □ 

Corollary 5.5.8 Every finite connected graph possesses a spanning tree. 

Proof. This is a consequence of property (2) of Theorem 15.5.61 Indeed, if there is some edge, 
e E E, such that deleting e yields a connected graph, Gi, we consider Gi and repeat this 
deletion procedure. Eventually, we will get a minimal connected graph that must be a tree. 
□ 

An example of a spanning tree (shown in thicker lines) in a graph is shown in Figure 

Em 

An endpoint or leaf in a graph is a node of degree 1. 

Proposition 5.5.9 Every finite tree with m >2 nodes has at least two endpoints. 

Proof . By Theorem I5.5.6[ our tree has m — 1 edges and by the version of Proposition 15.2.51 
for undirected graphs, 

J2dG{u)=2{m-l). 

If we had dc{u) > 2 except for a single node uq, we would have 

^c/g(m) > 2m - 1, 
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Figure 5.13: An Arborescence, T2 

contradicting the above. □ 

Remark: A forest with m nodes and p connected components has m — p edges. Indeed, if 
each connected component has rrii nodes, then the total number of edges is 

(mi — 1) + (m2 — 1) + ■ ■ ■ + {mp — 1) = m — p. 

We now consider briefly directed versions of a tree. 

Definition 5.5.10 Given a digraph, G = {V,E,s,t), a node, a G is a root (resp. anti- 
root) iff for every node u & V, there is a path from a to u (resp. there is a path from u to 
a). A digraph with at least two nodes is an arborescence with root a iff 

1. The node a is a root of G 

2. G is a tree (as an undirected graph). 

A digraph with at least two nodes is an anti- arborescence with anti-root a iff 

1. The node a is an anti-root of G 

2. G is a tree (as an undirected graph). 

Note that orienting the edges in a tree does not necessarily yield an arborescence (or an 
anti-arborescence). Also, if we reverse the orientation of the arcs of an arborescence we get 
an anti-arborescence. An arborescence is shown is Figure [5.131 

There is a version of Theorem 15.5.61 giving several equivalent characterizations of an 
arborescence. The proof of this theorem is left as an exercise to the reader. 

Theorem 5.5.11 Let G be a finite digraph with m = \V\ > 2 nodes. The following proper- 
ties characterize arborescences with root a: 
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(1) G is a tree (as undirected graph) with root a. 

(2) For every u eV , there is a unique path from a to u. 

(3) G has a as a root and is minimal for this property (if we delete any edge of G, then a 
is not a root any longer). 



(4) G is connected (as undirected graph) and moreover 

(*) 



dQ^a) = 

dQ{u) = 1, for all u E V, u ^ a. 

(5) G is acyclic (as undirected graph) and the properties (*) are satisfied. 

(6) G is acyclic (as undirected graph) and has a as a root. 

(7) G has a as a root and has m — 1 arcs. 

5.6 Minimum (or Maximum) Weight Spanning Trees 

For a certain class of problems, it is necessary to consider undirected graphs (without loops) 
whose edges are assigned a "cost" or "weight". 

Definition 5.6.1 A weighted graph is a finite graph without loops, G — (V, E, st), together 
with a function, c: — > M, called a weight function (or cost function). We will denote a 
weighted graph by (G, c). Given any set of edges, E' C E, we define the weight (or cost) of 
E' by 

c(£;') = J]c(e). 

eeE' 

Given a weighted graph, (G, c), an important problem is to find a spanning tree, T such 
that c(T) is maximum (or minimum). This problem is called the maximal weight spanning 
tree (resp. minimal weight spanning tree). Actually, it is easy to see that any algorithm 
solving any one of the two problems can be converted to an algorithm solving the other 
problem. For example, if we can solve the maximal weight spanning tree, we can solve the 
mimimal weight spanning tree by replacing every weight, c(e), by — c(e), and by looking for 
a spanning tree, T, that is a maximal spanning tree, since 

min c(T) = — max— c(T). 

TCG TCG 

There are several algorithms for finding such spanning trees, including one due to Kruskal 
and another one due to Prim. The fastest known algorithm at the present is due to Bernard 
Ghazelle (1999). 
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Figure 5.14: The set Cg associated with an edge e E G — T 

Since every spannning tree of a given graph, G = {V,E,st), has the same number of 
edges (namely, — 1), adding the same constant to the weight of every edge does not affect 
the maximal nature a spanning tree, that is, the set of maximal weight spanning trees is 
preserved. Therefore, we may assume that all the weights are non-negative. 

In order to justify the correctness of Kruskal's algorithm, we need two definitions. Let 
G = (V, E, st) be any connected weighted graph and let T be any spanning tree of G. For 
every edge, e E E — T, let C^ be the set of edges belonging to the unique chain joining 
the endpoints of e (the vertices in st{e)). For example, in the graph shown in Figure 15. 14^ 
the set C{8,ii} associated with the edge {8, 11} (shown as a dashed line) corresponds to the 
following set of edges (shown as dotted lines) in T: 

C{8,11} = {{8,5},{5,9},{9,11}}. 

Also, given any edge, e G T, observe that the result of deleting e yields a graph denoted 
T — e consisting of two disjoint subtrees of T. We let f2e be the set of edges, e' e G — T, 
such that if st(e') = {u, f }, then u and v belong to the two distinct connected components of 
T — {e}. For example, in Figure EISl deleting the edge {5, 9} yields the set of edges (shown 
as dotted lines) 

%,9} = {{1, 2}, {5, 2}, {5, 6}, {8, 9}, {8, 11}}. 

Observe that in the first case, deleting any edge from Cg and adding the edge e E E — T 
yields a new spanning tree and in the second case, deleting any edge e G T and adding any 
edge in Qe also yields a new spanning tree. These observations are crucial ingredients in the 
proof of the following theorem: 



5.6. MINIMUM (OR MAXIMUM) WEIGHT SPANNING TREES 



181 




Figure 5.15: The set il{5,9} obtained by deleting the edge {5, 9} from the spanning tree. 



Theorem 5.6.2 Let {G, c) be any connected weighted graph and let T he any spanning tree 
ofG. (1) The tree T is a maximal weight spanning tree iff any of the following (equivalent) 
conditions hold: 



(i) For every e E E — T, 

c(e) < min c(e') 

e'eCe 

(a) For every e & T, 

c(e) > maxc(e'). 



(2) The tree T is a minimal weight spanning tree iff any of the following (equivalent) 
conditions hold: 



(i) For every ee E — T, 



c(e) > maxc(e') 

e'eCe 



(a) For every e & T, 

c(e) < min c(e'). 

Proof. (1) First, assume that T is a maximal weight spanning tree. Observe that 

(a) For any e & E — T and any e' e Ce, the graph T' = (V, (T U {e}) — {e'}) is acyclic and 
has |K| — 1 edges, so it is a spanning tree. Then, (i) must hold, as otherwise we would 
have c(T') > c(T), contradicting the maximality of T. 
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(b) For any e e T and any e' e Qe, the graph T' = (V, (T U {e'}) — {e}) is connected and 
has 1 1^1 — 1 edges, so it is a spanning tree. Then, (ii) must hold, as otherwise we would 
have c(T") > c{T), contradicting the maximality of T. 

Let us now assume that (i) holds. We proceed by contradiction. Let T be a spanning tree 
satisfying condition (i) and assume there is another spanning tree, T', with c(T') > c(T). 
Since there are only finitely many spanning trees of G, we may assume that T' is maximal. 
Consider any edge e E T' — T and let st{e) = {u,v}. In T, there is a unique chain, Ce, 
joining u and v and this chain must contain some edge, e' e T, joining the two connected 
components of T' — e, that is, e' G Qe- As (i) holds, we get c(e) < c(e'). However, as T" is 
maximal, (ii) holds (as we just proved), so c(e) > c(e'). Therefore, we get 

c(e)=c(e'). 

Consequently, if we form the graph T2 = (T' U {e'}) — {e}), wc see that T2 is a spanning tree 
having some edge from T and c(T2) = c{T'). Wc can repeat this process of edge substitution 
with T2 and T and so on. Ultimately, we will obtain the tree T with the weight c(T') > c(T), 
which is absurd. Therefore, T is indeed maximal. 

Finally, assume that (ii) holds. The proof is analogous to the previous proof: We pick 
some edge e' & T — T' and e is some edge in Qe' belonging to the chain joining the endpoints 
of e' in T'. 

(2) The proof of (2) is analogous to the proof of (1) but uses 2(a) and 2(b) instead of 
1(a) and 1(b). □ 

We are now in the position to present a version of Kruskal's algorithm and to prove its 
correctness. Here is a version of Kruskal's algorithm for finding a minimal weight spanning 
tree using criterion 2(a). Let n be the number of edges of the weighted graph, (G, c), where 
G = {V,E,st). 



function Kruskal{{G,c): weighted graph): tree 
begin 

Sort the edges in non-decreasing order of weights: 
c(ei) < c(e2) < • • • < c(en); 
T := 0; 

for i := 1 to n do 

if (V, T U {ci}) is acyclic then T := T U {ci} 
endif 
endfor; 
Kruskal :— T 
end 
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We admit that the above description of Kruskal's algorithm is a bit sketchy as we have 
not exphcitly specified how we check that adding an edge to a tree preserves acychcity. On 
the other hand, it is quite easy to prove the correctness of the above algorithm. It is not 
difficult to refine the above "naive" algorithm to make it totally explicit but this involves a 
good choice of data structures. We leave these considerations to an algorithms course. 

Clearly, the graph T returned by the algorithm is acyclic, but why is it connected? Well, 
suppose T is not connected and consider two of its connected components, say Ti and T2. 
Being acyclic and connected, Ti and T2 are trees. Now, as G itself it connected, for any 
node of Ti and nay node of T2, there is some chain connecting these nodes. Consider such a 
chain, C, of minimal length. Then, as Ti is a tree, the first edge, ej, of C cannot belong to 
Ti since otherwise, we would get an even shorter chain connecting Ti and T2 by deleting ej. 
Furthermore, ej does not belong to any other connected component of T, as these connected 
components are pairwise disjoint. But then, T + ej is acyclic, which means that when we 
considered the addition of edge ej to the current graph, T^^\ the test should have been 
positive and ej should have been added to T^^\ Therefore, T is connected and so, it is a 
spanning tree. Now, observe that as the edges are sorted in non-decreasing order of weight, 
condition 2(a) is enforced and by Theorem 15. 6. 2[ T is a minimal weight spanning tree. 

We can easily design a version of Kruskal's algorithm based on considion 2(b). This time, 
we sort the edges in non-increasing order of weights and, starting with G, we attempt to 
delete each edge, ej, as long as the remaining graph is still connected. We leave the design 
of this algorithm as an exercise to the reader. 

Prim's algorithm is based on a rather different observation. For any node, v E V, let f/„ 
be the set of edges incident with v that are not loops, 

U^ = {eeE\ve st{e), st{e) G [Vf}. 

Choose in f/„ some edge of minimum weight which we will (ambiguously) denote by e{v). 

Proposition 5.6.3 Let {G,c) be a connected weighted graph with G = (y,E,st). For every 
vertex, v & V , there is a minimum weight spanning tree, T, so that e{v) G T. 

Proof. Let T' be a minimum weight spanning tree of G and assume that e{v) ^ T'. Let G 
be the chain in T' that joins the endpoints of e{v) and let e the edge of G incident with v. 
Then, the graph T" = (V, (T' U {e(f )}) — {e}) is a spanning tree of weight less that or equal 
to the weight of T' and as T' has minimum weight, do does T". By construction, e{v) G T". 
□ 

Prim's algorithm uses an edge-contraction operation described below: 

Definition 5.6.4 Let G = (V, E, st) be a graph, and let e G -E be some edge which is not a 
loop, i.e., st(e) = {u,v}, with u^v. The graph, Ce(G), obtained by contracting the edge e 
is the graph obtained by merging u and v into a single node and deleting e. More precisely, 
GJyG) = {(y — {u, v}) U {w}, E — {e}, ste), where w is any new node not in V and where 



184 



CHAPTER 5. GRAPHS, BASIC NOTIONS 



1. ste{e') = st{e') iff M ^ st{e') and v ^ st{e') 

2. ste(e') = {w,z} iff st(e') = {u,z}, with z ^ st(e) 

3. ste(e') = {z,w} iff st(e') = {z,v}, with z ^ st(e) 

4. ste(e') = ^ iff st(e') = {u,v}. 

Proposition 5.6.5 Let G = {V,E,st) be a graph. For any edge, e & E, the graph G is a 
tree iff Ge{G) is a tree. 

Proof . Proposition 15.6.51 follows from Theorem 15.5.61 Observe that G is connected iff Ge{G) 
is connected. Moreover, if G is a tree, the number of nodes of Ge{G) is rig = |\^| — 1 and 
the number of edges of Ge{G) is rUe = \E\ — 1. Since \E\ = \V\ — 1, we get me = — 1 
and Ge{G) is a tree. Conversely, if Ge{G) is a tree, then me = — 1, \V\ = Ue + 1 and 
\E\ = rrie + 1, so m = n — 1 and G is a tree. □ 

Here is a "naive" version of Prim's algorithm. 

function Prim{{G = {V, E, st),c): weighted graph): tree 
begin 
T:=0; 

while |V^| > 2 do 

pick any vertex v E V; 

pick any edge (not a loop), e, in Uy of minimum weight; 

r:=TU{e}; G := Ce(G) 
endwhile; 
Prim := T 
end 

The correctness of Prim's algorithm is an immediate consequence of Proposition 15.6.31 
and Proposition I5.6.5| the details are left to the reader. 

5.7 F-Cycles, Cocycles, Cotrees, Flows and Tensions 

In this section, we take a closer look at the structure of cycles in a finite graph, G. It turns 
out that there is a dual notion to that of a cycle, the notion of a cocycle. Assuming any 
orientation of our graph, it is possible to associate a vector space, JF, to the set of cycles in 
G, another vector space, T, to the set of cocycles in G, and these vector spaces are mutually 
orthogonal (for the usual inner product). Furthermore, these vector spaces do not depend 
on the orientation chosen, up to isomorphism. In fact, if G has m nodes, n edges and p 
connected components, we will prove that dim = n — m + p and dim T = m — p. These 
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vector spaces are the flows and the tensions of the graph G, and these notions are important 
in combinatorial optimization and the study of networks. This chapter assumes some basic 
knowledge of linear algebra. 

Recall that if G is a directed graph, then a cycle, C, is a closed simple chain, which 
means that C is a sequence of the form C = {uq, ei, ui, 62, U2, ■ ■ ■ , Wn-i, Cn, Un), where n > 1; 
Ui E V; Ci E E and 

uo = Un] {s{ei),t{ei)} = {ui_i,Ui}, 1 <i <n and ^ ej for all i ^ j. 

The cycle, C, induces the sets C+ and C~ where consists of the edges whose orientation 
agrees with the order of traversal induced by C and where C~ consists of the edges whose 
orientation is the inverse of the order of traversal induced by C. More precisely, 

= {ci e C \ s{ei) = Ui_i, t{ei) = Ui} 

and 

= {ei e C \ s{ei) = Ui, t{ei) = Ui^i}. 

For the rest of this section, we assume that G is a finite graph and that its edges are 
named, ei, . . . , eJl]. 

Definition 5.7.1 Given any finite directed graph, G, with n edges, to every cycle, G, is 
associated a representative vector, 7(G) G M", defined so that for every i, with I < i < n, 

( +1 if e, G G+ 
7(G), = <^ -1 ife, gG- 
[0 if Ci ^ G. 

For example, if G = Gg is the graph of Figure 15. 16^ the cycle 

C = (f3, 67, V4, ee, V5, 65, V2, ei, vi, 62, fs) 

corresponds to the vector 

7(G) = (-1,1,0,0,-1,-1,1). 

Observe that distinct cycles may yield the same representative vector unless they are 
elementary cycles. For example, the cycles 

Ci = {V2, 65, V5, 66, V4, 64, V2, Ci, f 1, 62, V3, 63, V2) 

and 

G2 = {V2, ei, Vi, 62, V3, 63, V2, 65, V5, 66, V4, 64, V2) 

^We use boldface notation for the edges in E in order to avoid confusion with the edges occurring in a 
cycle or in a chain; those are denoted in italic. 
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Figure 5.16: Graph Gs 



yield the same representative vector 

7=(-l,l,l,l,l,l,0). 

In order to obtain a bijection between representative vectors and "cycles" , we introduce the 
notion of a "F-cycle" (some authors redefine the notion of cycle and call "cycle" what we 
call a F-cycle, but we find this practice confusing). 

Definition 5.7.2 Given a finite directed graph, G = {V,E,s,t), a T-cycle is any set of 
edges, r = r+ U r~, such that there is some cycle, C, in G with r+ = and = C^; we 
say that the cycle, C, induces the T-cycle, T. The representative vector, 7(r), (for short, 7) 
associated with F is the vector, 7(C), from Definition 15. 7. 11 where C is any cycle inducing F. 
We say that a F-cycle, F, is a T-circuit iff either F"*" = or F~ = and that F is elementary 
iff F arises from an elementary cycle. 

Remarks: 

1. Given a F-cycle, F = F+ U F^, we have the subgraphs G^ = {V,T~^,s,t) and G^ = 
(y, F~, s, t). Then, for every u E V, we have 

cij+ (u) - (u) - (u) + d^_ (u) = 0. 

2. If F is an elementary F-cycle, then every vertex of the graph (V, F, s, t) has degree or 
2. 

3. When the context is clear and no confusion may arise, we often drop the "F" is F-cycle 
and simply use the term "cycle'. 

Proposition 5.7.3 // G is any finite directed graph, then any T-cycle, T, is the disjoint 
union of elementary T-cycles. 
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Proof. This is an immediate consequence of Proposition 15.4.51 □ 

Corollary 5.7.4 If G is any finite directed graph, then any T-cycle, T, is elementary iff it 
is minimal, i.e., if there is no T-cycle, T' , such that F' C F and F' ^ F. 

We now consider a concept which will turn out to be dual to the notion of F-cycle. 

Definition 5.7.5 Let G be a finite directed graph, G = {V, E, s,t), with n edges. For any 
subset of nodes, Y CV, define the sets of edges, fl~^iY) and Q~(Y) by 

n+{Y) = {eeE\ s{e) G F, t(e) ^ Y} 
n-{Y) = {e e E \ s{e) ^ Y, t{e) e Y} 
Q(Y) = Q+{Y)UQ-{Y). 

Any set, Q, of edges of the form Q = ^l(Y), for some set of nodes, F C V^, is called a cocycle 
(or cutset). To every, cocycle, Q, we associate the representative vector, uj{Q) G M", defined 
so that 

( +1 if a G fi+ 

u}{Q)i = < -1 if G n- 
[o iici^n, 

with 1 < i < n. We also write uj{Y) for uj{Q) when i7 = ^1{Y). If either ^^(Y) = or 
Q^(Y) = 0, then Q is called a cocircuit and an elementary cocycle (or bond) is a minimal 
cocycle (i.e., there is no cocycle, Q', such that Q' (1 Q and fi' 7^ fi). 

In the graph, Gg, of Figure [5?T6l 

n = {eg} U {61,62,66} 

is a cocycle induced by the set of nodes, Y = {t>2, V3, V4} and it corresponds to the vector 

a;(r]) = (-1,-1,0,0,1,-1,0). 
This is not an elementary cocycle because 

^' = {es} U {66} 

is also a cocycle (induced by Y' = {vi,V2,V3,Vi}). Observe that Q' is a minimal cocycle, so 
it is an elementary cocycle. Observe that the inner product 

7(Ci) ■ uj{n) = (-1, 1, 1, 1, 1, 1, 0) ■ (-1, -1, 0, 0, 1, -1, 0) = 1 - 1 + + + 1 - 1 + = 

is zero. This is a general property that we will prove shortly. 

Observe that a cocycle, fl, is the set of edges of G that join the vertices in a set, Y, 
to the vertices in its complement, V — Y. Consequently, deletetion of all the edges in Q 
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Figure 5.17: A coycle, Q, equal to the edge set of a cycle, F 

will increase the number of connected components of G. We say that is a cutset of G. 
Generally, a set of edges, K <Z E, is a cutset of G if the graph (V, E — K,s, t) has more 
connected components than G. 

It should be noted that a cocycle, Q = Q{Y), may coincide with the set of edges of some 
cycle, F. For example, in the graph displayed in Figure \5A7\ the cocycle, Q = ^{{l, 3, 5, 7}), 
shown in thicker lines, is equal to the set of edges of the cycle, 

(1, 2), (2, 3), (3, 4), (4, 1), (5, 6), (6, 7), (7, 8), (8, 5). 

If the edges of the graph are listed in the order 

(1, 2), (2, 3), (3, 4), (4, 1), (5, 6), (6, 7), (7, 8), (8, 5), (1, 5), (2, 6), (3, 7), (4, 8) 

the reader should check that the vectors 

7 = (1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0) G and lu = (1, -1, 1, -1, 1, -1, 1, -1, 0, 0, 0, 0, ) G T 

correspond to F and Q, respectively. 

We now give several characterizations of elementary cocycles. 

Proposition 5.7.6 Given a finite directed graph, G = (V, E, s, t), a set of edges, S ^ E, is 
an elementary cocycle iff it is a minimal cutset. 

Proof. We already observed that every cocycle is a cutset. Furthermore, we claim that every 
cutset contains a cocyle. To prove this, it is enough to consider a minimal cutset, S, and to 
prove the following satement: 

Claim. Any minimal cutset, S, is the set of edges of G that join two nonempty sets of 
vertices, Yi and Y2, such that 

(i) Y,nY2 = 0; 
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(ii) Fi U F2 = C, some connected component of G; 

(iii) The subgraphs Gyi and Gyj, induced by Yi and Y2 are connected. 

Indeed, if S* is a minimal cutset, it disconnects a unique connected component of G, say C. 
Let Ci, . . . , Cfc be the connected components of the graph, G—S, obtained from G by deleting 
the edges in 5*. Adding any edge, e & S, to G — S, must connect two components of G since 
otherwise, S — {e} would disconnect G, contradicting the minimality of G. Furthermore, 
k = 2, since otherwise, again, S — {e} would disconnect G. Then, if Yi is the set of nodes of 
Gi and Y2 is the set of nodes of G2, it is clear that the Claim holds. 

Now, if 5* is a minimal cutset, the above argument shows that S contains a cocyle and 
this cocycle must be elementary (i.e., minimal as a cocycle) as it is a cutset. Conversely, 
if 5* is an elementary cocycle, i.e., minimal as a cocycle, it must be a minimal cutset since 
otherwise, S would contain a strictly smaller cutset which would then contain a cocycle 
strictly contained in S. □ 

Proposition 5.7.7 Given a finite directed graph, G = (V, E, s,t), a set of edges, S ^ E, is 
an elementary cocycle iff S is the set of edges of G that join two nonempty sets of vertices, 
Yi and Y2, such that 

(i) Y,r\Y2 = 0; 

(ii) Y1UY2 = G, some connected component of G; 

(iii) The subgraphs Gyi and Gy^, induced by Yi and Y2 are connected. 

Proof. It is clear that if 5* satisfies (i)-(iii), then 5* is a minimal cutset and by Proposition 
15.7.71 it is an elementary cocycle. 

Let us first assume that G is connected and that S = ^(Y) is an elementary cocycle, i.e., 
is minimal as a cocycle. If we let Yi = Y and Y2 = X — Yi, it is clear that (i) and (ii) are 
satisfied. If Gyi or Gy^ is not connected, then if Z is a connected component of one of these 
two graphs, we see that Q{Z) is a cocycle strictly contained in S' = ^{Yi), a contradiction. 
Therefore, (iii) also holds. If G is not connected, as 5* is a minimal cocycle it is a minimal 
cutset and so, it is contained in some connected component, G, of G and we apply the above 
argument to G. □ 

The following proposition is the analog of Proposition 15 . 7. 31 for cocycle: 

Proposition 5.7.8 Given a finite directed graph, G = {V,E,s,t), every cocycle, Q = ^(Y), 
is the disjoint union of elementary cocycles. 

Proof . We give two proofs. 

Proof 1 : (Claude Berge) Let Yl, . . . , Y^ be the connected components of the subgraph 
of G induced by Y . Then, it is obvious that 



n(Y) = n{Yi)u---un{Yk), 
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where the fl{Yi) are pairwise disjoint. So, it is enough to show that each fl(Yi) is the union 
of disjoint elementary cycles. 

Let C be the connected component of G that contains Yi and let Ci, . . . ,Cm be the 
connected components of the subgraph, C — Y, obtained from C by deleting the nodes in Yi 
and the edges incident to these nodes. Observe that the set of edges that are deleted when 
the nodes in Yi are deleted is the union of fl(Yi) and the edges of the connected subgraph 
induced by Yi. As a consequence, we see that 

= fi(Ci)u---ufi(c^), 

where fi(Cfc) is the set of edges joining Ck and nodes from Yi in the connected subgraph 
induced by the nodes in Yi U Uj^fc^'j- By Proposition 15. 7. 8[ the set fl{Ck) is an elementary 
cocycle and it is clear that the sets fi(Cfc) are pairwise disjoint since the Ck are disjoint. 

Proof 2: (Michel Sakarovitch) Let Q = ^{Y) be a cocycle in G. Now, is a cutset and 
we can pick some minimal cocycle, Qi = ^{Z), contained in Qi. We proceed by induction 
on |f2 — fii |. If f2 = fii, we are done. Otherwise, we claim that Ei = Q — Qi is a cutset in G. 
If not, let e be any edge in Ei, we may assume that a = s(e) e Y and b = t(e) eV — Y . As 
El is not a cutset, there is a chain, C, from a to 6 in (V, E — Ei, s,t) and as is a cutset, 
this chain must contain some edge e', in Q, so C = Ci{x, e', y)C2, where Ci is a chain from a 
to X and C2 is a chain from y to b. Then, since C has its edges in E — Ei and Ei = Q — Qi, 
we must have e' G ^i. We may assume that x = s(e') G Z and y = t{e') E V — Z. But, we 
have the chain, C^{a,e,b)C2, joining x and y in {V,E — Qi), a contradiction. Therefore, 
El is indeed a cutset of G. Now, there is some minimal cocycle, Q2, contained in Ei, and if 
we let E2 = El — Qi, we can show as we just did that E2 is a cutset of G with \E2\ < \Ei. 
Thus, we finish the proof by applying the induction hypothesis to E2. □ 

We now prove the key property of orthogonality between cycles and cocycles. 

Proposition 5.7.9 Given any finite directed graph, G = (y,E,s,t), if '~f = 7(C) is the 
representative vector of any V-cycle, V = r(C), and u = uj(Y) is the representative vector 
of any cocycle, Q = fl(Y), then 

n 

7 ■ = ^ 'jiUi = 0, 
1=1 

i.e., 7 and u are orthogonal. (Here, \E\ = n.) 

Proof . Recall that F = C^ U C^ , where C is a cycle in G, say 

C = (Mo,ei,Mi, . . . ,Mfc-i,efc,Mfc), with Uk = Uq. 
Then, by definition, we see that 

^ . = |c+ n vl+{y)\ - \c+ n vl-{y)\ - \c- n n+{Y)\ + \c- n 9.-{Y)\. (*) 
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As we traverse the cycle, C, when we traverse the edge between and Ut {I < i < k), 
we note that 



In other words, every time we traverse an edge coming out from Y, its contribution to (*) 
is +1 and every time we traverse an edge coming into Y its contribution to (*) is —1. After 
traversing the cycle C entirely, we must have come out from Y as many times as we came 
into Y, so these contributions must cancel out. □. 

Note that Proposition 15.7.91 implies that |r n f2| is even. 

Definition 5.7.10 Given any finite digraph, G = (V, E, s, t), where E = {ei, . . . , e„}, the 
subspace, J-'{G), of spanned by all vectors, 7(r), where F is any F-cycle, is called the 
cycle space of G or flow space of G and the subspace, T{G), of M" spanned by all vectors, 
uj{fl), where fl is any cocycle, is called the cocycle space of G or tension space of G (or cut 
space of G) . 

When no confusion is possible, we write for J-'{G) and T for T{G). Thus, T is the 
space consisting of all linear combinations ^^^1=1 ^ili representative vectors of F-cycles, 
7j and T is the the space consisting of all linear combinations Yli=i '^i^i '^^ representative 
vectors of cocycles, Ui, with aj e M. Proposition 15.7.91 says that the spaces T and T are 
mutually orthogonal. 

Remark: The seemingly odd terminology "flow space" and "tension space" will be explained 
later. 

Our next goal will be to determine the dimensions of JF and T in terms of the number 
of edges, the number of nodes and the number of connected components of G and to give a 
convenient method for finding bases of and T. For this, we will use spanning trees and 
their dual, cotrees. But first, we will need a crucial theorem that also plays an important 
role in the theory of flows in networks. 

Theorem 5.7.11 (Arc Coloring Lemma; Minty [I960]) Let G = {V,E,s,t), be a finite 
directed graph and assume that the edges of G are colored either in black, red or green. Pick 
any edge, e, and color it black. Then, exactly one of two possibilities may occur: 

(1) There is an elementary cycle containing e whose edges are only red or black with all 
the black edges oriented in the same direction; 

(2) There is an elementary cocycle containing e whose edges are only green or black with 
all the black edges oriented in the same direction. 



(c+nf]+(r))u(c-nf]-(r)) iff 
(c+ni7-(r))u(c-nf]+(r)) iff 



Ui^i eY, UieV -Y 
Ui^i eV - Y, meY. 



Proof . Let a = s{e) and b = t{e). Apply the following procedure for making nodes: 
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Intitially, only b is marked. 

while there is some marked node x and some unmarked node y with 

either a black edge, e', with {x,y) = (s(e'), t(e')) or 

a red edge, e', with {x,y) = {s(e'), t(e')} 

then mark y; arc(y) = e' 
endwhile 

When the marking algorithm stops, exactly one of the following two cases occurs: 

(i) Node a has been marked. Let e' = arc(a) be the edge that caused a to be marked and 
let X be the other endpoint of e'. If x = b, we found an elementary cycle satisfying (i). 
If not, let e" = arc(x) and let y be the other endpoint of e" and continue in the same 
manner. This procedure will stop with b and yields the chain, C, from 6 to a along 
which nodes have been marked. This chain must be elementary because every edge in 
it was used once to mark some node (check that the set of edges used for the marking 
is a tree). If we add the edge, e, to the chain, C, we obtain an elementary cycle, F 
whose edges are colored black or red and with all edges colored black oriented in the 
same direction due to the marking scheme. It is impossible to have a cocycle whose 
edges are colored black or green containing e because it would have been impossible to 
conduct the marking through this cocycle and a would not have been marked. 

(ii) Node a has not been marked. Let Y be the set of unmarked nodes. The set ^{Y) 
is a cocycle whose edges are colored green or black containing e with all black edges 
in Q~^{Y). This cocycle is the disjoint of elementary cocycles (by Proposition 15.7.81) 
and one of these elementary cocycles contains e. If a cycle with black or red edges 
containing e with all black edges oriented in the same direction existed, then a would 
have been marked, a contradiction. □ 

Corollary 5.7.12 Every edge of a finite directed graph, G, belongs either to an elementary 
circuit or to an elementary cocircuit but not both. 

Proof . Color all edges black and apply Theorem 15.7.111 □ 

Although Minty's Theorem looks more like an amusing fact than a deep result, it is 
actually a rather powerful theorem. For example, we will see in Section 15.101 that Minty's 
Theorem can be used to prove the "hard part" of the Max-flow Min-cut Theorem (Theorem 
I5.10.7p . an important theorem that has many applications. Here are a few more applications 
of Theorem 15.7.111 

Proposition 5.7.13 Let G be a finite connected directed graph with at lest one edge. Then, 
the following conditions are equivalent: G is strongly connected iff 

(i) G is strongly connected. 
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(a) Every edge belongs to some circuit. 
(Hi) G has no cocircuit. 

Proof . {i) => {ii). If x and y are the endpoints of any edge, e, in G, as G is strongly 
connected, there is an elementary path from y to x and thus, an elementary circuit through 
e. 

(a) =^ [Hi). This follows from Corollary 15.7.121 

{Hi) =^ (i). Assume that G is not strongly connected and let Y' and Y" be two strongly 
connected components linked by some edge, e, and let a = s(e) and b = t{e), with a G F' 
and b G Y". The edge e does not belong to any circuit since otherwise, a and b would belong 
to the same strongly connected component. Thus, by Corollary I5.7.12[ the edge e should 
belong to some cocircuit, a contradiction. □ 

In order to determine the dimension of the cycle space, T, we will use spanning trees. Let 
us assume that G is connected since otherwise the same reasoning applies to the connected 
components of G. If T is any spanning tree of G, we know from Theorem I5.5.6[ part (4), 
that adding any edge, e E E — T, (called a chord of T) will create a (unique) cycle. We will 
see shortly that the vectors associated with these cycles form a basis of the Cycle space. We 
can find a basis of the cocycle space by considering sets of edges of the form E — T, where 
T is a spanning tree. Such sets of edges are called cotrees. 

Definition 5.7.14 Let G be a finite directed connected graph, G = {V, E, s,t). A spanning 
subgraph, {V, K, s, t), is a cotree iff (V^, E — K,s, t) is a spanning tree. 

Cotrees are characterized in the following proposition: 

Proposition 5.7.15 Let G be a finite directed connected graph, G = {V,E,s,t). If E is 
partitioned into two subsets, T and K, (i.e.,TUK = E;Tr\K = (l};T,K^^), then the 
following conditions ar equivalent: 

(1) {y, T, s, t) is tree. 

(2) {y,K,s,t) is a cotree. 

(3) {y,K,s,t) contains no elementary coycles of G and upon addition of any edge, e 
it does contain an elementary cocycle of G. 

Proof . By definition of a cotree, (1) and (2) are equivalent, so we will prove the equivalence 
of (1) and (3). 

(1) =^ (3). We claim that {y,K,s,t) contains no elementary coycles of G. Otherwise, 
K would contain some elementary coycle, r(A), of G and then no chain in the tree (V, T, s, t) 
would connect A and V — E, a. contradiction. 
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Next, for any edge, e G T, observe that (V, T — {e}, s, t) has two connected components, 
say A and B and then, VL{A) is an elementary cocycle contained in (V, K U {e}, s, t) (in fact, 
it is easy to see that it is the only one). Therefore, (3) holds 

(3) =^ (1). We need to prove that (V,T, s,t) is tree. First, we show that (V,T, s,t) has 
no cycles. Let e G T be any edge; color e black; color all edges in T — {e} red; color all edges 
in K = E — T green. By (3), by adding e to K, we find an elementary cocycle of black or 
green edges that contains e. Thus, there is no cycle of red or black edges containing e. As e 
is arbitrary, there are no cycles in T. 

Finally, we prove that (V,T, s,t) is connected. Pick any edge, e G and color it black; 
color edges in T red; color edges in K — {e} green. Since G has no cocycle of black and green 
edges containing e, there is a cycle of black or red edges containing e. Therefore, T U {e} 
has a cycle, which means that there is a path from any two nodes in T. □ 

We are now ready for the main theorem of this section. 

Theorem 5.7.16 Let G be a finite directed graph, G = (V, E, s, t) and assume that \E\ = n, 
\V\ = m and that G has p connected components. Then, the cycle space, T , and the cocycle 
space, T, are subspaces of M" of dimensions dim = n — m + p and dim T = m — p and 
T = is the orthogonal complement of T . Furthermore, if Ci, . . . ,Cp are the connected 
components of G, bases of J-" and T can be found as follows: 

(1) Let Ti, . . . ,Tp, be any spanning trees in Ci, . . . ,Cp. For each spanning tree, Ti, form 
all the elementary cycles, Fj^g, obtained by adding any chord, e E Ci — T^, to Tj. Then, 
the vectors 7,^6 = liXi,e) form a basis of J-'. 

(2) For any spanning tree, Ti, as above, let Ki = Ci — Ti be the corresponding cotree. For 
every edge, e &Ti (called a twig), there is a unique elementary cocycle, fij^g? contained 
in Ki U {e}. Then, the vectors Ui^e = i^(^«,e) form a basis of T . 

Proof . We know from Proposition 15.7.91 that JF and T are orthogonal. Thus, 

dim JF + dim T <n. 

Let us follow the procedure specified in (1). Let Ci = {Ei, Vi),he the i-th connected compo- 
nent of G and let rii = \Ei\ and |V^| = rrii, so that ni + ■ ■ ■ + rip = n and mi + ■ ■ ■ + rrip = m. 
For any spanning tree, T,, for Ci, recall that Tj has rrii — l edges and so, \Ei — Ti\ = rii — mi + l. 
If Cj^i, . . . , ei^m-mi+i the edges in Ei — Ti, then the vectors 

must be linearly independent, because 7^,6^,^ = l{^i,ei^j) and the elementary cycle, Fj^gij, 
contains the edge, Cjj, that none of the other Fj e^fe contain for k 7^ j. So, we get 

(ni — mi + 1) + ■ ■ ■ + {jip — nip + l) = n — m + p< dim JF. 
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Let us now follow the procedure specified in (2). For every spanning tree, Tj, let 
Ci,!, . . . , Ci^mi-i be the edges in Tj. We know from proposition 15.7.151 that adding any edge, 
Cij to Ci — Ti determines a unique elementary cocycle, ^i.e^.j , containing Cij and the vectors 

must be linearly independent since the elementary cocycle, fij e. ., contains the edge, eij, 
that none of the other fij e, ^ contain for k ^ j. So, we get 

(mi — 1) + ■ ■ ■ + (mp — 1) = m — p < dimT. 

But then, n < dim JF + dim T, and since we also have dim JF + dim T < n, we get 

dim J-' = n — m + p and dim T = m — p. 

Since the vectors produced in (1) and (2) are linearly independent and in each case, their 
number is equal to the dimension of the space to which they belong, they are bases of these 
spaces. □ 

Since dim^ = n — m + p and dimT = m — p do not depend on the orientation of G, 
we conclude that the spaces JF and T are uniquely determined by G, independently of the 
orientation of G, up to isomorphism. The number n — m + p is called the cyclomatic number 
of G and m — p is called the cocydomatic number of G. 

Remarks: 

1. Some authors, including Harary [2H] and Diestel [13], define the vector spaces JF and T 
over the two-element field, F2 = {0, 1}. The same dimensions are obtained for and T 
and JF and T still orthogonal. On the other hand, because 1 + 1 = 0, some interesting 
phenomena happen. For example, orientation is irrelevant, the sum of two cycles (or 
cocycles) is their symmetric difference and the space JF fi T is not necessarily reduced 
to the trivial space, (0). The space jFpiT is called the bicycle space. The bicycle space 
induces a partition of the edges of a graph called the principal tripartition. For more 
on this, Godsil and Royle [23], Sections 14.15 an 14.16 (and Chapter 14). 

2. For those who know homology, of course, p = dimi^o, the dimension of the zero-th 
homology group and n — m + p = dim Hi , the dimension of the first homology group 
of G viewed as a topological space. Usually, the notation used is 60 = dimi^o and 
bi = dimifi (the first two Betti numbers). Then, the above equation can be rewritten 
as 

m — n = Bq — hi, 
which is just the formula for the Euler-Poincare characteristic. 

Figure I5.18[ shows an unoriented graph (a cube) and a cocycle, f2, which is also a cycle, 
r, shown in thick lines (i.e., a bicycle, over the field F2). However, as we saw in the example 
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Figure 5.18: A bicycle in a graph (a cube) 

from Figure EUTJ for any orientation of the cube, the vectors, 7 and uj, corresponding to F 
and Q are different (and orthogonal). 

Let us illustrate the procedures for constructing bases of and T on the graph Gg. 
Figure \5A9\ shows a spanning tree, T and a cotree, K for Gg. 

We have n = 7;m = 5; p = 1, and so, dimjF = 7 — 5 + 1 = 3 and dimT = 5 — 1 = 4. 
If we add successively the edges 62, eg, and ey to the spanning tree, T, we get the three 
elementary cycles shown in Figure IS. 2UI with thicker lines. 

If we add successively the edges ei, 63, 64 and 65 to the cotree, K, we get the four 
elementary cocycles shown in Figures 15.211 and 15.221 with thicker lines. 

Given any node, v & V, in a graph, G, for simplicity of notation let us denote the 
cocycle Q{{v}) by fl{v). Similarly, we will write fl~^{v) for fl'^{{v}); fl~{v) for fl~{{v}), and 
similarly for the the vectors, uj{{v}), etc. It turns our that vectors of the form uj{v) generate 
the cocycle space and this has important consequences. 

Proposition 5.7.17 Given any finite directed graph, G = {V,E,s,t), for every cocycle, 
Q = fl(Y), we have 

Consequently, the vectors of the form uj{v), with v & V , generate the cocycle space, T. 

Proof . For any edge, e G -E, if a = s(e) and h = t{e), observe that 

{+1 if t> = a 
— 1 if V = b 
if t> 7^ a, 6. 

As a consequently, if we evaluate ^t,gyi^('y), we find that 

/ \ f +1 a a eV and b eV - Y 

\^uj{v)] = < -1 ifaeV -Y andbeY 
\v€Y / e I if a,beY or a,beV -Y, 
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Figure 5.19: Graph Gg; A Spanning Tree, T; A Cotree, K 



Fi : r2 : : 




^^5 



Figure 5.20: A Cycle Basis for Gs 




Figure 5.21: A Cocycle Basis for Gs 
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^4 : 




Figure 5.22: A Cocycle Basis for Gg (continued) 

which is exactly uj{Y)^. □ 

Proposition 15.7. ITI allows us to characterize flows (the vectors in JF) in an interesting way 
which also reveals the reason behind the terminology. 

Theorem 5.7.18 Given any finite directed graph, G = {V,E,s,t), a vector, f G M", is a 
flow in T iff" 

E [/(^)]- E [/(e)] = 0' /«^«^^ ^e^- (t) 

Proof. By Theorem I5.7.16[ we know that is the orthogonal complement of T. Thus, for 
any / G M", we have / GJFiff/-u; = for all G T. Moreover, Proposition 15.7.171 says 
that T is generated by the vectors of the form u^v), where w G so / G iff / • uj^v) = 
for all V E V. But, (f) is exactly the assertion that / ■ u^v) = and the theorem is proved. 
□ 

Equation (f) justifies the terminology of "flow" for the elements of the space JF. Indeed, 
a flow, f, in a (directed) graph, G = (V, E, s, t), is defined as a function, / : — >• R, and we 
say that a flow is conservative (Kirchhoff 's first law) iff for every node, v E V, the total flow, 
Seen+(i.)[/(e)]) coming into the vertex, v, is equal to the total flow, Segn-(t)) [/(e)]; coming 
out of that vertex. This is exactly what equation (f) says. 

We can also characterize tensions as follows: 

Theorem 5.7.19 Given any finite simple directed graph, G = {V,E,s,t), for any, t G , 
we have: 

(1) The vector, t, is a tension in T iff for every elementary cycle, P = P"*" U P", we have 



E W)] - E W)] = 0- 

eer+ eGr- 



(*) 
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(2) If G has no parallel edges (and no loops), then t is a tension in T iff the following 
condition holds: There is a function, t^-.V-^M., called a "potential function" , such 
that 

t(e) = 7r(t(e))-7r(s(e)), (**) 

for every e & E. 

Proof . (1) The equation, (*), asserts that 7(r) ■ t = for every elementary cycle, F. Since 
every cycle is the disjoint union of elementary cycles, the vectors of the form 7(r) generate 
the flow space, J-', and by Theorem 15. 7. 16^ the tension space T is the orthogonal complement 
of JF, so t is a tension iff (*) holds. 

(2) Assume a potential function, vr: — > M, exists, let T = {vo,ei,vi, . . . ,Vk-i,ek,Vk), 
with Vk = Vq, be an elementary cycle and and let 7 = 7(r). We have 

7it(ei) = n{vi) - ni^vo) 
72^(62) = n{v2) - n{vi) 

7fc-lt(efe_i) = 7T{Vk_i) - 7l{Vk^2) 



and we see that when we add up both sides of these equations that we get (*): 

e6r+ eer- 

Let us now assume that (*) holds for every elementary cycle and let t G T be any tension. 
Consider the following procedure for assigning a value, 7r(f ), to every vertex, v & V, so that 
(**) is satisfied. Pick any vertex, vq, and assign it the value, 7r(t>o) = 0. 

Now, for every vertex, v ^ V, that has not yet been assigned a value, do the following: 

1. If there is an edge, e = {u,v), with tt{u) already determined, set 

t{v) =t{u)+t{e); 

2. If there is an edge, e = {v,u), with 7r(M) already determined, set 

t{v) = t{u)-t{e). 

At the end of this process, all the nodes in the connected component of Vq will have received 
a value and we repeat this process for all the other connected components. However, we 
have to check that each nodes receives a unique value (given the choice of Vq). If some node. 
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V, is assigned two different values, 7ii{v) and 7i2{v), then there exist two chains, ai and (T2, 
from vq to V, and if C is the cycle (Jicr^, we have 

7(C) -t^O. 

However, any cycle is the disjoint union of elementary cycles, so there would be some ele- 
mentary cycle, r, with 

7(r) • i ^ 0, 

contradicting (*). Therefore, the function tt is indeed well-defined and, by construction, 
satisfies (**). □ 

Some of these results can be improved in various ways. For example, flows have what is 
called a "conformal decomposition" . 

Definition 5.7.20 Given any finite directed graph, G = {V,S,s,t), we say that a fiow, 
f & has a conformal decomposition, iff there are some cycles, ri,...,rk, such that if 
7i = 7(ri), then 

/ = Q;i7i H \- Oiklk, 

with 

1. ctj > 0, for i = 1, . . . , k; 

2. For any edge, e e if /(e) > (resp. /(e) < 0) and e e F^-, then e e F^ (resp. 

Proposition 5.7.21 Given any finite directed graph, G = {V,S,s,t), every flow, f E J-', 
has some conformal decomposition. In particular, if /(e) > for all e & E, then all the Fj 's 
are circuits. 

Proof . We proceed by induction on the number on nonzero components of /. First, note 
that f — has a trivial conformal decomposition. Next, let / e be a fiow and assume 
that every flow, /', having at least one more zero component than / has some conformal 
decomposition. Let G be the graph obtained by reversing the orientation of all edges, e, 
for which /(e) < and deleting all the edges for wich /(e) = 0. Observe that G has no 
cocircuit, as the inner product of any elementary cocircuit with any nonzero fiow cannot be 
zero. Hence, by the corollary to the Coloring Lemma, G has some circuit, C, and let F be a 
cycle of G corresponding to C. Let 

a — minjmin /(e), min —/(e)} > 0. 
eer+ eer- 

Then, the fiow 

/' = / - a7(r) 

has at least one more zero component than /. Thus, /' has some conformal decomposition 
and, by construction, / = /' + tt7(r) is a conformal decomposition of /. □ 

We now take a quick look at various matrices associated with a graph. 
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5.8 Incidence and Adjacency Matrices of a Graph 

In this section, we are assuming that our graphs are finite, directed, without loops and 
without parallel edges. 

Definition 5.8.1 Let G = (V, E) be a graph with V = {vi, . . . , v^} and E = {ei, . . . , e„}. 
The incidence matrix, D{G), of G, is the m x ra-matrix whose entries, dij, are 

( +1 if Vj = s{ej) 
V otherwise. 

Remark: The incidence matrix actually makes sense for a graph, G, with parallel edges but 
without loops. 

For simplicity of notation and when no confusion is possible, we write D instead of D{G). 

Since we assumed that G has no loops, observe that every column of D contains exactly 
two nonzero entries, +1 and —1. Also, the ith row of D is the vector, ct;(vj), representing 
the cocycle, fi(vj). For example, here is the incidence matrix of the graph Gg shown again 
in Figure 15^231 



/ 1 


1 














o\ 


-1 





-1 


-1 


1 











-1 


1 











1 











1 





-1 


-1 


Vo 











-1 


1 


0/ 



The incidence matrix, D, of a graph, G, represents a linear map from M" to called 
the incidence map (or boundary map and denoted by D (or d). For every e G -E, we have 



^(ej) = s(ej)-t(ej). 
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Remark: Sometimes, it is convenient to consider the vector space, Ci{G) = M.^ , of all 
functions, f : E ^ R, called the edge space of G and the vector space, Co{G) = MX, of all 
functions, g : V ^ M., called the vertex space of G. Obviously, Gi (G) is isomorphic to R" 
and Go{G) is isomorphic to M™. The transpose, , of D, is a linear map from Go{G) to 
Gi{G) also called the cohoundary map and often denoted by 5. Observe that 5{Y) = fi(Y) 
(viewing the subset, Y ^ V, ClS db vector in Go{G)). 

The spaces of flows and tensions can be recovered from the incidence matrix. 

Theorem 5.8.2 Given any finite graph, G, if D is the incidence matrix of G and andT 
are the spaces of flows and tensions on G, then 

(1) = Ker D; 

(2) T = '^D^. 

Futhermore, if G hasp connected components and m nodes, then 

rank D = m — p. 

Proof . We already observed that the ith row of D is the vector cij(vj) and we know from 
Theorem 15.7.181 that is exactly the set of vectors orthogonal to all vectors of the form 
cij(vj). Now, for any / G M"", 

/^(vi)-/\ 

A/= ; 

and so, JF = Ker D. Since the vectors co'(vj) generate T, the rows of D generate T, i.e.. 

From Theorem 15.7.161 we know that 

dim T = m — p 
and since we just proved that T = ^ , we get 

rank D = rank = m — p, 
which proves the last part of our theorem. □ 

Corollary 5.8.3 For any graph, G = {V, E, s, t), if\V\ = m, \E\ = n and G has p connected 
components, then the incidence matrix, D,ofG has rankn (i.e., the columns of D are linearly 
independent) iff J-' = (0) iff n = m — p. 

Proof . By Theorem 15.8.31 we have rankD = m — p. So, rankD = niSn = m— piS 
n — m + p = iS J-' = (0) (since dim T = n — m + p). □ 

The incidence matrix of a graph has another interesting property observed by Poincare. 
First, let us define a variant of triangular matrices. 
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Definition 5.8.4 An n x n (real or complex) matrix, A = (aij), is said to be pseudo- 
triangular and non-singular iff either 

(i) n = 1 and an 7^ 0, or 

(ii) n > 2 and A has some row, say k, with a unique nonzero entry, ah,k, such that the 
submatrix, B, obtained by deleting the h-th row and the k-th column from A is also 
pseudo-triangular and non-singular. 

It is easy to see a matrix defined as in Definition 15.8.41 can be transformed into a usual 
triangular matrix by permutation of its columns. 

Proposition 5.8.5 (Poincare, 1901) If D is the incidence matrix of a graph, then every 
square kx k nonsingular submatri^, B, of D is pseudo-triangular. Consequently, det{B) = 
+1, —1, or 0, for any square k x k submatrix, B, of D. 

Proof . We proceed by induction on k. The result is obvious for k = 1. 

Next, let -B be a square k x fc-sub matrix of D which is nonsingular, not pseudo-triangular 
and yet, every nonsingular h x /i-submatrix of B is pseudo-triangular if h < k. We know that 
every column of B has at most two nonzero entries (since every column of D contains two 
nonzero entries: +1 and —1). Also, as B is not pseudo-triangular (but nonsingular) every 
row of B contains at least two nonzero elements. But then, no row of B may contain three of 
more elements, because the number of nonzero slots in all columns is at most 2k and by the 
pigeonhole principle, we could fit 2fc -|- 1 objects in 2k slots, which is impossible. Therefore, 
every row of B contains exactly two nonzero entries. Again, the pigeonhole principle implies 
that every column also contains exactly two nonzero entries. But now, the nonzero entries 
in each column are +1 and —1, so if we add all the rows of 5, we get the zero vector, which 
shows that B is singular, a contradiction. Therefore, B is pseudo-triangular. 

Since the entries in D are +1, —1, 0, the above immediately implies that 
det{B) = +1, —1, or 0, for any square k x k submatrix, B, of D. □ 

A square matrix such. A, such that det(-B) = +1, —1, or 0, for any square kxk submatrix, 
B, of A is said to be totally unimodular. This is a very strong property of incidence matrices 
that has far reaching implications in the study of optimization problems for networks. 

Another important matrix associated to a graph is its adjacency matrix. 

Definition 5.8.6 Let G = (V, E) be a graph with V = {vi, . . . , v^}. The ajacency matrix, 
A{G), of G, is the m x m-matrix whose entries, aij, are 




^ Given any rax. n matrix, A = {ai j), li \ < h < m and 1 < fc < n, then a, h x fc-submatrix, B, of A is 
obtained by picking any k columns of A and then any h rows of this new matrix. 
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When no confusion is possible, we write A for A{G). Note that the matrix A is symmetric 
and an = 0. Here is the adjacency matrix of the graph Gg shown in Figure 15.231 



A 



/O 1 1 0\ 
10 111 
110 10 
110 1 

\0 1 1 0/ 



We have the following useful relationship between the incidence matrix and the adjacency 
matrix of a graph: 

Proposition 5.8.7 Given any graph, G, if D is the incidence matrix of G, A is the adja- 
cency matrix of G and A is the diagonal matrix such that An = (i(vj), the degree of node 
Vj, then 

DD^ = A-A. 

Consequently, DD^ is independent of the orientation of G and A — A is symmetric positive, 
semi- definite, i.e., the eigenvalues of A — A are real and non-negative. 

Proof . It is well-known that DDjj is the inner product of the ith row, di and the jth row, 
dj, of D. li i = j, then as 

+1 if s(efc) = Vj 
dik= i -I if t{ek) = Vj 
otherwise, 



we see that di - di = diyi). \ii ^ j, then di ■ dj ^ iff there is some edge, with s{ek) = Vj 
and t(efc) = Vj, in which case, di ■ dj = —1. Therefore, 



A- A, 



as claimed. Now, DD^ is obviously symmetric and it is well kown that its eigenvalues are 
non- negative (for example, see Gallier [19], Chapter 12). □ 



Remarks: 

1. The matrix, L = DD^ = A — A, is known as the Laplacian (matrix) of the graph, G. 
Another common notation for the matrix DD^ is Q. Since the colums of D contain 
exactly the two nonzero entries, +1 and —1, we see that the vector, 1, defined such 
that Ij = 1, is an eigenvector for the eigenvalue 0. 

2. If G is connected, then D has rank m — 1, so the rank of DD^ is also m — 1 and 
the other eigenvalues of DD^ besides are strictly positive. The smallest positive 
eigenvalue of L = DD^ has some remarkable properties. There is an area of graph 
theory overlapping (linear) algebra, called spectral graph theory that investigates the 
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properties of graphs in terms of the eigenvalues of its Laplacian matrix but this is 
beyond the scope of these notes. Some good references for algebraic graph theory 
include Biggs [5], Godsil and Royle [23] and Chung [9], for spectral graph theory. 

One of the classical and surprising results in algebraic graph theory is a formula that 
gives the number of spanning trees, t{G), of a connected graph, G, in terms of its 
Laplacian, L = DD^ . If J denotes the square matrix whose entries are all I's and if 
adj L denotes the adjoint matrix of L (the transpose of the matrix of cofactors of L), 
i.e. the matrix given by 

(adjL),, = (-iy+MetL(j,0, 

where i) is the matrix obtained by deleting the jth row and the i-column of L, 
then we have 

adjL = r(G)J. 

We also have 

t{G) = m-Met(J + L), 
where m is the number of nodes of G. 

3. As we already observed, the incidence matrix also makes sense for graphs with parallel 
edges and no loops. But now, in order for the equation DD^ = A — A to hold, we need 
to define A differently. We still have the same definition as before for the adjacency 
matrix but we can define the new matrix. A, such that 

Aij = \{ee E \ s{e) = v^, t{e) = Vj}|, 

i.e., Aij is the number of parallel edges between Vj and Vj. Then, we can check that 

DD^ = A-A. 

4. There are also versions of the adjacency matrix and of the incidence matrix for undi- 
rected graphs. In this case, D is no longer totally unimodular. 

5.9 Eulerian and Hamiltonian Cycles 

In this short section, we discuss two classical problems that go back to the very beginning 
of graph theory. These problems have to do with the existence of certain kinds of cycles in 
graphs. These problems come in two flavors depending whether the graphs are directed or 
not but there are only minor differences between the two versions and traditionally the focus 
is on undirected graphs. 

The first problems goes back to Euler and is usually known as the Konigsberg bridge 
problem. In 1736, the town of Konigsberg had seven bridges joining four areas of land. 
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Figure 5.24: A graph modeling the Konigsberg bridge problem 



Euler was asked whether it is possible to find a cycle that crosses every bridge exactly once 
(and returns to the starting point). 

The graph shown in Figure 15.241 models the Konigsberg bridge problem. The nodes 
A^B^C^D correspond to four areas of land in Konigsberg and the edges to the seven bridges 
joining these areas of land. 

In fact, the problem is unsolvable, as shown by Euler, because some nodes do not have 
an even degree. We will now define the problem precisely and give a complete solution. 

Definition 5.9.1 Given a finite undirected graph, G = {V,E), (resp., a directed graph, 
G = {V, E, s,t)), an Euler cycle (or Euler tour) (resp. an Euler circuit) is a cycle in G that 
passes through every node and every edge (exactly once) (resp. a circuit in G that passes 
through every node and every edge (exactly once)). The Eulerian cycle (resp. circuit) 
Problem is the problem: Given a graph G, is there an Eulerian cycle (resp. circuit) in G7 

Theorem 5.9.2 (1) An undirected graph, G = (V, E), has an Eulerian cycle iff the following 
properties hold: 

(al) The graph G is connected. 

(hi) Every node has even degree. 

(2) A directed graph, G = (V, E, s, t), has an Eulerian circuit iff the following properties 
hold: 

(a2) The graph G is strongly connected. 

(h2) Every node has the same number of incoming and outgoing edges, i.e., d^{v) = d^(v), 
for all V eV. 

Proof . We prove (1) leaving (2) as an easy exercise (the proof of (2) is very similar to the 
proof of (1)). Clearly, if a Euler cycle exists, G is connected and since every edge is traversed 
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exactly once, every node is entered as many times as it is exited so the degree of every node 
is even. 

For the converse, observe that G must contain a cycle as otherwise, being connected, G 
would be a tree but we proved earlier that every tree has some node of degree 1. (If G is 
directed and strongly connected, then we know that every edge belongs to a circuit.) Let T 
be any cycle in G. We proceed by induction on the number of edges in G. If G has a single 
edge, clearly T = G and we are done. If G has no loops and G has two edges, again T = G 
and we are done. If G has no loops and no parallel edges and if G has three edges, then 
again, T = G. Now, consider the induction step. Assume T G and consider the graph 
G' = (y, E — T) . Let Gi, . . . ,Gp be the connected components of G'. Pick any connected 
component, Gi, of G'. Now, all nodes in Gi have even degree, Gi is connected and Gi has 
strictly fewer edges than G so, by the induction hypothesis, Gi contains an Euler cycle, Fj. 
But then, F and each Fj share some vertex and we can combine F and the Fj's to form an 
Euler cycle in G. □ 

There are iterative algorithms that will find an Euler cycle if one exists. It should also 
be noted that testing whether or not a graph has an Euler cycle is computationally quite an 
easy problem. This is not so for the Hamiltonian cycle problem described next. 

A game invented by Sir William Hamilton in 1859 uses a regular solid dodecahedron 
whose twenty vertices are labeled with the names of famous cities. The player is challenged 
to "travel around the world" by finding a circuit along the edges of the dodecahedron which 
passes through every city exactly once. 

In graphical terms, assuming an orientation of the edges between cities, the graph D 
shown in Figure [5.251 is a plane projection of a regular dodecahedron and we want to know if 
there is a Hamiltonian cycle in this directed graph (this is a directed version of the problem). 

Finding a Hamiltonian cycle in this graph does not appear to be so easy! A solution is 
shown in Figure \572B\ below: 

Definition 5.9.3 Given any undirected graph, G, (resp. directed graph, G) a Hamiltonian 
cycle in G (resp. Hamiltonian circuit in G) is a cycle that passes though every vertex 
of G exactly once (resp. circuit that passes though every vertex of G exactly once). The 
Hamiltonian cycle (resp. circuit) problem is to decide whether a graph, G has a Hamiltonian 
cycle (resp. Hamiltonian circuit). 

Unfortunately, no theorem analogous to Theorem 15.9.21 is known for Hamiltonian cycles. 
In fact, the Hamiltonian cycle problem is known to be NP-complete and so far, appears to 
be a computationally hard problem (of exponential time complexity). Here is a proposition 
that may be used to prove that certain graphs are not Hamiltonian. However, there are 
graphs satisfying the condition of that proposition that are not Hamiltonian! 
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Proposition 5.9.4 // a graph, G = {V,E), possesses a Hamiltonian cycle then, for every 
nonempty set, S, of nodes, if G{V — S) is the induced subgraph of G generated hyV — S and 
if c{G{V — S)) is the number of connected components of G{V — S), then 

c{G{V-S))<\S\. 

Proof . Let F be a Hamiltonian cycle in G and let G be the graph G = (V, F). If we delete 
k vertices we can't cut a cycle into more than k pieces and so 

c{G{V-S)) < \S\. 

However, we also have ^ 

c{G{V-S)) <c{G{V-S)), 

which proves the proposition. □ 

5.10 Network Flow Problems; The Max-Flow Min-Cut 
Theorem 

The network flow problem is a perfect example of a problem which is important practically 
but also theoretically because in both cases it has unexpected applications. In this sec- 
tion, we solve the network flow problem using some of the notions from Section [STTl First, 
let us describe the kinds of graphs that we are dealing with, usually called networks (or 
transportation networks or flow networks). 

Definition 5.10.1 A network {or flow network) is a quadruple, = {G,c,Vs, St), where G 
is a finite diagraph, G = {V, E, s, t), without loops, c: E ^ M+, is a function called capacity 
function assigning a capacity, c(e) > 0, (or cost or weight), to every edge, e E E, and 
Vs, Vt G V are two (distinct) distinguished nodesjf] Moreover, we assume that there are no 
edges incoming into Vg {dQ{vs) = 0), which is called the source and that there are no edges 
outgoing from Vt {d'^{vt) = 0), which is called the terminal (or sink). 

An example of a network is showed in Figure 15.271 with the capacity of each edge showed 
within parentheses. 

Intuitively, we can think of the edges of a network as conduits for fluid, or wires for 
electricity, or highways for vehicle, etc., and the capacity of each edge is the maximum 
amount of "flow" that can pass through that edge. The purpose of a network is to carry 
"flow" , defined as follows: 

Definition 5.10.2 Given a network, A^ = {G, c, Vs, Vt), a flow in N is a function, / : E M., 
such that the following conditions hold: 

•^Most books use the notation s and t for and Vf. Sorry, s and t are already used in the definition of a 
digraph! 
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Figure 5.27: A network, 



(1) (Conservation of flow) 

E /(^) = E /(^)' all G r - {v,,vt} 

t{e)=v s{e)=v 

(2) (Admissibility of flow) 

< /(e) < c(e), for all e G ^ 
Given any two sets of nodes, S,T CV, let 



f{S,T) 



J2 fie) and c(5,T)= J] c(e). 



s(e)eS, t(e)eT 



s(e)e5, i(e)eT 



When S = {u} or T = {v}, we write /(m, T) for f{{u}, T) and /(S*, v) for /(S*, {f }) (similarly, 
we write c{u,T) for c({m},T) and c{S,v) for 0(5, {f})). The net flow out of S is defined as 
f{S, S) — /(S*, S) (where S = V — S). The value, \ f\ (or v{f) ) of the flow f is the quantity 

\f\=f{Vs,V-{Vs}). 

We can now state the 

Network Flow Problem: Find a flow, /, in A^, for which the value, |/|, is maximum (we 
call such a flow a maximum flow). 

Figure [5.281 shows a flow in the network A^, with value, |/| = 3. This is not a maximum 
flow, as the reader should check (the maximum flow value is 4). 

Remarks: 

1. For any set of edges, £ ^ E,\et 



ci£) = ^c(e). 
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Figure 5.28: A flow in the network, 
Then, note that the net flow out of S can also be expressed as 

fin^is)) - fin-{s)) = fis,s) - f(s, s). 

Now, recall that Q{S) = Q^{S) U Q^{S)) is a cocycle (see Deflnition I5.7.5p . So if we 
deflne the value, f{Q{S)), of the cocycle, ^(5), to be 

f{n{s)) = f{n^{S))-f{n-{S)), 

the net flow through S is the value of the cocycle, Q{S). 

2. By deflnition, c{S,S) = c(fi+(S)). 

3. Since G has no loops, there are no edges from u to itself, so 

fiu,V-{u}) = fiu,V) 

and similarly, 

f{V-{v},v) = f{V,v). 

4. Some authors (for example, Wilf |43]) do not require the distinguished node, Vs, to 
be a source and the distinguished node, Vt, to be a sink. This makes essentially no 
difference but if so, the value of the flow / must be deflned as 

I/I = f{v,, V - {vs}) - f{y - K}, Vs) = f{vs, V) - f{v, Vs). 

Intuitively, because flow conservation holds for every node except Vg and Vt, the net flow, 
/(V, Vt), into the sink should be equal to the net flow, f{vs, V) our of the source, Vs. This is 
indeed true and follows from the following proposition: 
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Proposition 5.10.3 Given a network, N = {G,c,Vs,Vt), for any flow, f , in N and for any 

subset, S V , if Vs & S and Vt ^ S, then the net flow through S has the same value, namely 
I/I, that is 

I/I = finiS)) = f{S,S) - f(S, S) < c{S,S) = c(fi+(5)). 

In particular, 

\f\ = fivs,V) = fiV,Vt). 

Proof. Recall that |/| = f{vs,V). Now, for any node, v E S — {vs}, since v ^ Vt, the 
equation 

t{e)=v s{e)=v 

holds and we see that 

I/I = f{vs, v) = j2iJ2 /(^) - E /(^)) = ^) - /(^' ^)) = /(^' ^) - /(^' 

v€S s{e)=v t(e)=v v£S 

However, V = S U S, so 

I/I = f{S,V)-J{V,S) 

= f{s,sus)-fj^sus,s)_ 

= fiS,S) + fiS,S)-fiS,S)-fiS,S) 
= f{S,S)-f{S,S), 

as claimed. Since the capacity of every edge is non-negative, it is obvious that 

I/I = f{S,S) - f{S,S) < f{S,S) < c{S,S) = c(fi+(5)), 

since a flow is admissible. Finally, if we set = \^ — {vt}, we get 

f{S,S)-f{S,S) = f{V,Vt) 

and so, \ f\ = f{v,,V) = f{V,Vt). □ 

Proposition 15.10.3) shows that the sets of edges, Q~^{S), with Vs E S and vt ^ S, play a 
very special role. Indeed, as a corollary of Proposition I5.10.3[ we see that the value any flow 
in N is bounded by the capacity, c(f2"''(S')), of the set fl^{S), for any S with Vg E S and 
Vt ^ S. This suggests the following definition: 

Definition 5.10.4 Given a network, N = {G, c, Vs, Vt), a cut separating Vs and Vt, for short 
a Vg-Vf-cut, is any subset of edges, C = Q~^{W), where is a subset of V with Vg E W and 
Vt ^ W. The capacity of a Vg-Vt-cut, C, is 

c{C) = c{n^{w))= E 

een+(H/) 
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Figure 5.29: A maximum flow and a minimum cut in the network, 

Remark: Some authors, including Papadimitriou and Steightz [33J and Will [43j, define 
a Vg-Vt-cut as a pair (ly, ly), where W is a subset of V with with Vs E W and Vt ^ W. 
This definition is clearly equivalent to our definition above, which is due to Sakarovitch [38j. 
We have a slight prerefence for Definition 15.10.41 because it places the emphasis on edges 
as opposed to nodes. Indeed, the intuition behind f g-Vt-cuts is that any flow from Vs to Vt 
must pass through some edge of any Vs-Vt-cnt. Thus, it is not surprising that the capacity 
of Vg-Vt-cuts places a restriction on how much flow can be sent from Vg to Vt- 

We can rephrase Proposition 15.10.31 as follows: 

Proposition 5.10.5 The maximum value of any flow, f, in N is bounded by the minimum 
capacity, c{C), of any Vg-Vf-cut, C, in N , i.e., 

max I /I < min c(C). 

Proposition 15. 10^5] is half of the so-called Max-flow Min-cut Theorem. The other half of 
this theorem says that the above inequality is indeed an equality. That is, there is actually 
some Vg-Vt-cni, C, whose capacity, c(C), is the maximum value of the flow in A^. 

A Vg-Vt-cut of minimum capacity is called a minimum Vg-Vt-cut, for short, a minimum 
cut. 

An example of a minimum cut is shown in Figure 15.291 where 

C = n+{{Vg, V2}) = {(VgVi), iv2Vt)}, 

these two edges being shown as thicker lines. The capacity of this cut is 4 and a maximum 
flow is also shown in Figure 15.291 

What we intend to do next is to prove the celebrated "Max-flow, Min-cut Theorem" (due 
to Ford and Fulkerson, 1957) and then to give an algorithm (also due to Ford and Fulkerson) 
for finding a maximum flow, provided some reasonable assumptions on the capacity function. 
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In preparation for this, we present a handy trick (found both in Berge [3] and Sakarovitch 
[38]). the return edge. 

Recall that one of the consequences of Proposition 15.10.31 is that the net flow out from 
Vs is equal to the net flow into Vf. Thus, if we add a new edge, Cr, called the return 
edge, to G, obtaining the graph G (and the network A^), we see that any flow, /, in N 
satisfying condition (1) of Definition 15. 10.21 yields a genuine flow, /, in (a flow according 
to Definition 15.7.10^ by Theorem 15.7.181) . such that /(e) = /(e) for every edge of G and 
fi^r) = I/I- Consequently, the Network fiow problem is equivalent to find a (genuine) fiow 
in such that f{er) is maximum. Another advantage of this formulation is that all the 
results on fiows from Section 15.71 can be applied directly to A^. To simplify the notation, as 
/ extends /, let us also use thw notation / for /. Now, if D is the indicence matrix of G 
(again, we use the simpler notation, D, instead of D), we know that / is a fiow iff 

Df = 0. 

Therefore, the network flow problem can be stated as a linear programing problem as follows: 

Maximize z = f{er) 

subject to the linear constraints 

Df = 

< / 

f < c, 

where we view / as a vector in M""*"^, with n = \E{G)\. 

Consequently, we obtain the existence of maximal flows, a fact which is not immediately 
obvious. 

Proposition 5.10.6 Given any network, N = [G,c,Vs,Vt) , there is some flow, f, of maxi- 
mum value. 

Proof. If we go back to the formulation of the Max- flow problem as a linear program, we 
see that the set 

C = {x e I < s < c} n Ker D 

is compact, as the intersection of a compact subset and a closed subset of IR"^'^ (in fact, G 
is also convex) and nonempty, as (the zero vector) is a flow. But then, the projection, 
7t: X ^ x{er), is a continuous function, vr: C — * M, on a nonempty compact, so it achieves 
its maximum value for some f ^ G. Such an / is a flow on A^ with maximal value. □ 

Now that we know that maximum flows exist, it remains to prove that a maximal flow 
is realized by some minimal cut to complete the Max-flow, Min-cut Theorem of Ford and 
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Fulkerson. This can be done in various ways usually using some version of an algorithm due 
to Ford and Fulkerson. Such proofs can be found in Papadimitriou and Steiglitz [3l], Wilf 
[13], Cameron [8] and Sakarovitch [38j. 

Sakarovitch makes the interesting observation (given as an exercise) that the Arc Coloring 
Lemma due to Minty (Theorem I5.7.1ip yields a simple proof of the part of the Max-flow, 
Min-cut Theorem that we seek to establish (See [38], Chapter 4, Exercise 1, page 105). 
Therefore, we choose to present such a proof since it is rather original and quite elegant. 

Theorem 5.10.7 (Max-Flow, Min-Cut Theorem (Ford and Fulkerson)) For any network, 
N = {G,c,Vs,Vt), the maximum value, \f\, of any flow, f, in N is equal to the minimum 
capacity, c{C), of any Vg-Vf-cut, C, in N . 

Proof . By Proposition I5.10!B| we already have half of our theorem. By Proposition I5.10!B| 
we know that some maximum flow, say /, exists. It remains to show that there is some 
fs-frcut, C, such that |/| = c(C). 

We proceed as follows: 

Form the graph, G = (y,EU {er},s,t) from G = (y,E,s,t), with s{er) = Vt and 
t(e.r) = Vg. Then, form the graph, G = {V,E,'s,t), whose edges are defined as follows: 

(a) Cr G E; s'{er) = s{er), t{er) = t(er); 

(b) If e e E and < /(e) < c(e), then e e E; s(e) = s{e), t{e) = t{e); 

(c) If e e E and /(e) = 0, then e e E; s(e) = s(e), t{e) = t(e); 

(d) If e G E and /(e) = c(e), then e E E, with s(e) = t(e) and t{e) = s{e). 

In order to apply Minty's Theorem, we color all edges constructed in (a), (c) and (d) in 
black and all edges constructed in (b) in red and we pick e,. as the distinguished edge. Now, 
apply Minty's Lemma. We have two possibilities: 

1. There is an elementary cycle, F, in G, with all black edges oriented the same way. 
Since Cr is incoming into Vg, the direction of the cycle is from from Vg to Vt, so e^ G F"*". 
This implies that all edges of type (d), e G E, have an orientation consistent with the 
direction of the cycle. Now, F is also a cycle in G and, in G, each edge, e E E, with 
/(e) = c(e) is oriented in the inverse direction of the cycle, i.e, e G F~^ in G. Also, all 
edges of type (c), e G E, with /(e) = 0, are oriented in the direction of the cycle, i.e., 
e G F'*' in G. We also have e^ G F"^ in G. 

We show that the value of the flow, |/|, can be increased. Since < /(e) < c(e) for 
every red edge, /(e) = for every edge of type (c) in F"*", /(e) = c(e) for every edge of 
type (d) in F~, and since all capacities are strictly positive, if we let 

6i = min{c(e) - /(e)} 
eer+ 

52 = min{/(e)} 
eer- 
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and 

5 = min{5i, ^2}, 

then 6 > 0. We can increase the flow, /, in N, by adding 6 to /(e) for every edge 
e G r+ (including edges of type (c) for which /(e) = 0) and subtracting S from /(e) 
for every edge e G F" (including edges of type (d) for which /(e) = c(e)) obtaining a 
flow, /' such that 

|/'| = /(e.) + 5=|/|+5>|/|, 

as e^ G F"*", contradicting the maximahty of /. Therefore, we conclude that alternative 
(1) is impossible and we must have the second alternative: 

2. There is an elementary cocycle, ^Iq{W), in G with all edges black and oriented in the 
same direction (there are no green edges). Since e^ G flQ(W), either Vs & W or Vt E W 
(but not both). In the second case {vt G W), we have e^ G Qi^iW) and Vg G W. Then, 

(j 

consider Q'^{W) = Q'^(W), with Vs G W. Thus, we are reduced to the case where 
Vs G W. 

If Vs G W, then e^ G ^^(W) and since all edges are black, ^If^CW) = Q.^(W), in G. 

However, as every edge, e G -E, of type (d) corresponds to an inverse edge, e E E, we 
see that l^g(Vr) defines a cocycle, ^^iW) = n±{W) U nz(W), with 

n+(W) = {eeE\ s{e) G W} 
n-~{W) = {eeE\ t{e) G W}. 

Moreover, by construction, /(e) = c(e) for all e G VL%{W), /(e) = for all 

e G Vt~^{W) — {e^}, and /(e^) = |/|. We say that the edges of the cocycle VL^{W) are 

saturated. Consequently, C = Qt(^W) is a fg-ft-cut in with 

c(C) = /(e.) = |/|, 

establishing our theorem. □ 

It is interesting that the proof in part (1) of Theorem 15.10.71 contains the main idea 
behind the algorithm of Ford and Fulkerson that we now describe. 

The main idea is to look for an (elementary) chain from Vs to Vt so that together with the 
return edge, e^, we obtain a cycle, F, such that the edges in F satisfy the following properties: 

(1) 61 = min,er+{c(e) - /(e)} > 0; 

(2) 52 = min,er-{/(e)}>0. 
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Such a chain is called a flow augmenting chain. Then, if we let S — mm{Si,S2}, we can 
increase the value of the flow by adding S to /(e) for every edge e e F"^ (including the edge, 
Cr, which belongs to F"*") and subtracting S from /(e) for all edges e G F~. This way, we 
get a new flow, /', whose value is |/'| = |/| + 5. Indeed, /' = / + 57(F), where 7(F) is the 
flow associated with the cycle, F. The algorithms goes through rounds each consisting of 
two phases: During phase 1, a flow augmenting chain is found by the procedure findchain; 
During phase 2, the flow along the edges of the augmenting chain is increased using the 
function changeflow. 

During phase 1, the nodes of the augmenting chain are saved in the (set) variable, Y, 
and the edges of this chain are saved in the (set) variable, S. We assign the special capacity 
value 00 to e^, with the convention that 00 ± a = a and that a < 00 for all a e M. 

procedure findchain{N: network; e^: edge; Y: node set; S: edge set; S; real; /; flow) 
begin 

5 S{vs) :~ 00; Y :— {vs}', 
while {vt ^ y) A (5 > 0) do 

if there is an edge e with s{e) G Y. t{e) ^ Y and /(e) < e(e) then 

Y:=YU {t{e)}; S{t{e)) := e; 6{t{e)) := min{5(s(e)), c(e) - /(e)} 
else 

if there is an edge e with t{e) G Y, s(e) ^ Y and /(e) > then 

y := y U {s{e)}; £{s{e)) := e; S{s{e)) := mm{5{t{e)), /(e)} 
else 5 :— (no new arc can be traversed) 
endif 
endif 
endwhile; 

if Vt eY then S :— S{vt) endif 
end 

Here is now the procedure to update the flow: 

procedure changeflow{N: network; e^: edge; S: edge set; 5: real; /; flow) 
begin 

u := Vt; f{er) := /(e^) + 5; 
while u ^ Vs do e := S{u); 

ii u = t{e) then /(e) := /(e) + 5; u :— s(e); 

else /(e) := /(e) - S; u = t{e) 

endif 
endwhile 
end 

Finally, the algorithm maxflow is given below: 
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procedure max flow {N : network; e^: edge; Y: set of nodes; £\ set of edges; /; flow) 
begin 

for each e E E do /(e) := enfdor; 
repeat until 5 = 

findchain{N, e,., Y,£,6,f)] 
if 5 > then 

changeflow{N, er, S, S, f) 
endif 
endrepeat 
end 

The reader should run the algorithm maxflow on the network of Figure 15.271 to verify 
that the maximum flow shown in Figure 15.291 is indeed found, with Y = {vs, V2} when the 
algorithm stops. 

The correctness of the algorithm maxflow is easy to prove. 

Theorem 5.10.8 If the algorithm, maxflow, terminates and during the last round through 
findchain the node Vt is not marked, then the flow, f , returned by the algorithm is a maximum 
flow. 

Proof. Observe that if Y is the set of nodes returned when maxflow halts, then Vs € Y, 
Vt ^Y and 

1. If e G Q~^(Y), then /(e) = c(e), as otherwise, procedure findchain would have added 
t(e) to Y; 

2. If e G Q^{Y), then /(e) = 0, as otherwise, procedure findchain would have added s(e) 
to Y. 

But then, as in the end of the proof of Theorem 15.10.71 we see that the edges of the coycle 
r2(y) are saturated and we know that Q^{Y) is a minimal cut and that |/| = c{Q^{Y)) is 
maximal. □ 

We still have to show that the algorithm terminates but there is a catch. Indeed, the 
version of the Ford and Fulkerson algorithm that we just presented may not terminate if the 
capacities are irrational! Moreover, in the limit, the flow found by the algorithm may not 
be maximum! An example of this bad behavior due to Ford and Fulkerson is reproduced 
in Wilf |13] (Chapter 3, Section 5). However, we can prove the following termination result 
which, for all practical purposes, is good enough, since only rational numbers can be stored 
by a computer. 

Theorem 5.10.9 Given a network, N, if all the capacities are multiple of some number, X, 
then the algorithm, maxflow, always terminates. In particular, the algorithm maxflow always 
terminates if the capacites are rational (or integral). 
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Proof. The number S will always be a multiple of A, so /(e,.) will increase by at least A 
during each iteration. Thus, eventually, the value of a minimal cut, which is a multiple of A, 
will be reached. □ 

If all the capacities are integers, an easy induction yields the following useful and non- 
trivial proposition: 

Proposition 5.10.10 Given a network, N, if all the capacities are integers, then the algo- 
rithm maxfiow outputs a maximum flow, f : E N, such that the flow in every edge is an 
integer. 

Remark: Proposition 15.10.101 only asserts that some maximum flow is of the form 

f : E ^ N. In general, there is more than one maximum flow and other maximum flows may 

not have integer values on all edges. 

Theorem 15 . 1 . 91 is good news but it is also bad news from the point of view of complexity. 
Indeed, the present version of the Ford and Fulkerson algorithm has a running time that 
depends on capacities and so, it can be very bad. 

There are various ways of getting around this difficulty to find algorithms that do not 
depend on capacities and quite a few researchers have studied this problem. An excellent 
discussion of the progress in network flow algorithms can be found in Wilf (Chapter 3). 

A fairly simple modiflcation of the Ford and Fulkerson algorithm consists in looking for 
flow augmenting chains of shortest length. To explain this algorithm we need the concept of 
residual network, which is a useful tool in any case. Given a network, = {G, c, s, t) and 
given any flow, /, the residual network, Nj = {Gf,Cf,Vf,Vt) is deflned as follows: 

1. = V- 

2. For every edge, e E E, ii /(e) < c(e), then e+ G Ef, s/(e+) = s(e), tf{e~^) = t(e) and 
c/(e+) = c(e) — /(e); the edge e+ is called a forward edge; 

3. For every edge, e E E, if /(e) > 0, then e~ G Ef, s/(e^) = t(e), tf{e~) = s(e) 
and c/(e~) = /(e); the edge e~ is called a backward edge because it has the inverse 
orientation of the original edge, e E E; 

The capacity, c/(e^), of an edge e^ G Ej (with e = ±) is usually called the residual 
capacity of e^ Observe that the same edge, e, in G, will give rise to two edges e"*" and e~ 
(with the same set of endpoints but with opposite orientations) in if < /(e) < c(e). 
Thus, Gf has at most twice as many edges as G. Also, note that every edge, e E E, which 
is saturated, i.e., for which /(e) = c(e), does not survive in Gj. 

Observe that there is a one-to-one correspondence between (elementary) flow augmenting 
chains in the original graph, G, and (elementary) flow augmenting paths inGf. Furthermore, 
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Figure 5.30: A poor choice of augmenting paths yields a slow method 



in order to check that an elementary path, tt, from Vg to ft in is a flow augmenting path, 
all we have to do is to compute 

c/(7r) =m.in{c/(e')}, 

the bottleneck of the path, tt. Then, as before, we can update the flow, / in A^, to get the 
new flow, /', by setting 

/'(e) = /(e)+c/(7r), if e+ G vr 
/'(e) = /(e) - Cf{7r) if e" G tt, 
/'(e) = /(e) if eeE and e' i tt, 

for every edge e G -E. Note that the function, /^ : ^ M, defined by 

/^(e) = c/(7r), if e+ G vr 

/^(e) = -c/(7r) if e" G vr, 

/^(e) =0 if e G -E and e^ ^ vr, 

is a flow in with |/^| = c/(7r) and /' = / + /Ar,7r is a flow in A^, with |/'| = |/| + c/(7r) 
(same reasoning as before). Now, we can repeat this process: Compute the new residual 
graph, N ^1 from A^ and /', update the flow /' to get the new flow /" in A^, etc. 

The same reasoning as before shows that if we obtain a residual graph with no flow 
augmenting path from Vg to fj, then a maximum flow has been found. 

It should be noted that a poor choice of augmenting paths may cause the algorithm to 
perform a lot more steps than necessary. For example, if we consider the network shown in 
Figure 15.301 and if we pick the flow augmenting paths in the residual graphs to be alterna- 
tively {vs,v\,V2-,Vt) and (f^, f2, fi, fi), at each step, we only increase the flow by 1, so it will 
take 200 steps to find a maximum fiow! 

One of the main advantages of using residual graphs is that they make is convenient to 
look for better strategies for picking fiow augmenting paths. For example, we can choose an 
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Figure 5.31: Construction of the residual graph, Nf, from N, round 1 




Figure 5.32: Construction of the residual graph, Nf, from N, round 2 

elementary flow augmenting path shortest length (for example, using breadth-first search). 
Then, it can be shown that this revised algorithm terminates in 0(|V^|-|£^|) steps (see Cormen, 
Leiserson, Rivest and Stein [10], Section 26.2, and Sakarovitch [38], Chapter 4, Exercise 5). 
Edmonds and Karp designed an algorithm running in time 0(|i?| ■ \V\'^) based on this idea 
(1972), see [lOj, Section 26.2. Another way of selecting "good" augmenting paths, the scaling 
Max-Flow algorithm, is described in Kleinberg and Tardos [52] (see Section 7.3). 

Here is an illustration of this faster algorithm, starting with the network, A^, shown in 
Figure 15.271 The sequence of residual network construction and fiow augmentation steps 
is shown in Figures 15. 3H 15.321 and 15.331 During the first two rounds, the augmented path 
chosen is shown in thicker lines. In the third and final round, there is no path from Vg to Vt 
in the residual graph, indicating that a maximum fiow has been found. 

Another idea originally due to Dinic (1970) is to use layered networks, see Wilf [43J 
(Sections 3.6-3.7) and Papadimitriou and Steiglitz [S^ (Chapter 9). An algorithm using 
layered networks running in time 0{y^) is given in the two references above. There are 
yet other faster algorithms, for instance "prefiow-push algorithms" also called "prefiow-push 
relabel algorithms", originally due to Goldberg. A preflow is a function, /: ^ M, that 
satisfies condition (2) of Definition 15.10.21 but which, instead of satisfying condition (1), 
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Figure 5.33: Construction of the residual graph, Nf, from N, round 3 



satisfies the inequahty 
(!') (Non-negativity of net flow) 

/(e) > E /(e) all G V - {vs,vt}, 

s{e)=v t{e)=v 

that is, the net flow out of v is non-negative. Now, the principle of all methods using prefiows 
is to augment a prefiow until it becomes a maximum flow. In order to do this, a labeling 
algorithm assigning a height. Algorithms of this type are discussed in Gormen, Leiserson, 
Rivest and Stein [10], Sections 26.4 and 26.5 and in Kleinberg and Tardos [32], Section 7.4. 

The Max-flow, Min-cut Theorem (Theorem I5.10.7p is a surprisingly powerful theorem 
in the sense that it can be used to prove a number of other results whose original proof is 
sometimes quite hard. Among these results, let us mention the maximum matching problem 
in a bipartite graph, discussed in Wilf [13] (Sections 3.8), Gormen, Leiserson, Rivest and Stein 
[To] (Section 26.3) Kleinberg and Tardos [32] (Section 7.5) and Gameron |8J (Ghapter 11, 
Section 10), finding the edge connectivity of a graph, discussed in Wilf [43J (Sections 3.8), 
and a beautiful theorem of Menger on edge-disjoint paths and Hall's Marriage Theorem, 
both discussed in Gameron [8j (Ghapter 11, Section 10). More problems that can be solved 
effectively using flow algorithms, including image segmentation, are discussed in Sections 
7.6-7.13 of Kleinberg and Tardos [32]. We only mention one of Menger's theorems, as it is 
particularly elegant. 

Theorem 5.10.11 (Menger) Given any finite digraph, G, for any two nodes, Vg andvt, the 
maximum number of pairwise edge-disjoint paths from Vg to Vt is equal to the the minimum 
number of edges in a Vs-Vt-separating set. (A a Vg-Vt-separating set in G is a set of edges, 
G , such every path from Vg to Vt uses some edge in G.) 

It is also possible to generalize the basic flow problem in which our flows, /, have the 
property that < /(e) < c(e) for every edge, e G -E, to channeled flows . This generalization 
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consists in adding another capacity function, b: E —* M., relaxing the condition that c(e) > 
for all e G -E, and in allowing flows such that condition (2) of Definition 15.10.21 is replaced 
by 

(2') (Admissibility of flow) 

b{e) < /(e) < c(e), for all e G ^ 

Now, the "flow" / = is no longer necessarily admissible and the channeled flow problem 
does not always have a solution. However, it is possible to characterize when it has a solution. 

Theorem 5.10.12 (Hoffman) A network, N = {G,b,c,Vs,Vt), has a channeled flow iff for 
every cocycle, ^{Y), ofG, we have 

J2 Ke)< E ^(^)- (t) 

Observe that the necessity of the condition of Theorem 15.10.121 is an immediate conse- 
quence of Proposition 15.7.91 That it is sufficient can be proved by modifying the algorithm 
maxflow or its version using residual networks. The principle of this method is to start with 
a flow, /, in that does not necessarily satisfy condition (2') and to gradually convert it 
to an admissible flow in N (if one exists) by applying the method for finding a maximum 
fiow to a modified version, iV, of N in which the capacities have been adjusted to that / is 
an admissible fiow in A^. Now, if a fiow, /, in A^ does not satisfy condition (2'), then there 
are some offending edges, e, for which either /(e) < 6(e) or /(e) > c(e). The new method 
makes sure that at the end of every (successful) round through the basic maxflow algorithm 
applied to the modified network, A^, some offending edge of A^ is no longer offending. 

Let / be a fiow in A^ and assume that e is an offending edge (i.e. either /(e) < 6(e) or 
/(e) > c(e)). Then, we construct the network, N{f,e), as follows: The capacity functions, b 
and c are given by 

6(e) if 6(e) < /(e) 
/(e) if /(e) < 6(e) 

c(e) if /(e) < c(e) 
/(e) if/(e)>c(e). 

We also add one new edge, e^, to A^ whose endpoints and capacities are determined by: 

1. If /(e) > c(e), then s(er) = t(e), t{er) = s(e), 6(6^) = and c(er) = /(e) — c(e). 

2. If /(e) < 6(e), then s{er) = ^(e), t(er) = t(e), 6(6^) = and c(er) = 6(e) — /(e). 



and 
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Now, observe that the original flow, /, in N extended so that f{er) = is a channeled 
flow in N{f,e) (i.e., conditions (1) and (2') are satisfled). Starting from the new network, 
N{f,e), apply the Max- flow algorithm, say using residual graphs, with the following small 
change in 2: 

1. For every edge, e e if /(e) < c(e), then e+ e £"/, Sf{e~^) = s(e), tf{e~^) = t{e) and 
Cf{e'^) — c(e) — /(e); the edge e'^ is called a forward edge; 

2. For every edge, e E E, if /(e) > 6(e), then e^ G Ef, Sf{e^) = t{e), tf{e~) = s(e) and 
Cf{e~) = /(e) — 6(e); the edge e^ is called a backward edge. 



Now, we consider augmenting paths from t{er) to s(ej.). For any such elementary path, 
TT, in N{f,e)f, as before we compute 

c/(7r) = inin{c/(e')}, 

the bottleneck of the path, tt, and we say that tt is a flow augmenting path iff C/(7r) > 0. 
Then, we can update the flow, / in A^(/, e), to get the new flow, /', by setting 

/'(e) = /(e) + c/(7r) if e" e tt, 
/'(e) = /(e) - c/(7r) if e" e tt, 
/'(e) = /(e) if eeE and ^ tt, 

for every edge e & E. 

We run the flow augmenting path procedure on N{f, e) and / until it terminates with a 
maximum flow, /. If we recall that the offending edge is e, then, there are four cases: 



1. /(^ > c(S). 

(a) When the Max- flow algorithm terminates, /(e^) = c(er) — f(e) — c(e). If so, 
deflne / as follows: 

?/ \ / /(e) — f(er) if e = e , . 

I /(e) ff e / e. 

It is clear that / is a flow in N and /(e) = c(e) (there are no elementary paths 
from tie) to s(e)). But then, e is not an offending edge for /, so we repeat the 
procedure of constructing the modifled network, etc. 

(b) When the Max-flow algorithm terminates, /(e,.) < c(er). The flow, /, deflned in 
(*) above is still a flow but the Max-flow algorithm must have terminated with 
a residual graph with no flow augmenting path from s(e) to t(e). Then, there is 
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a set of nodes, Y with s{e) ^ Y and t{e) ^ Y. Moreover, the way the Max-flow 
algorithm is designed implies that 

m > 

/(e) = c(e) > c(e) if e G Q^^Y) - 
/(e) = 6(e) < 6(e) if eeQ-{Y). 

As / also satisfies (*) above, we conclude that the cocycle condition (f) of Theorem 
Einiafailsfor fi(r). 

2. /(?) < 6(g). 

6(g) -/(i). If so, 

(**) 

It is clear that / is a flow in and /(e) = 6(e) (there are no elementary paths 
from s(e) to t{e)). But then, e is not an offending edge for /, so we repeat the 
procedure of constructing the modified network, etc. 

(b) When the Max-flow algorithm terminates, f(er) < c{er). The flow, /, defined in 
(**) above is still a flow but the Max-flow algorithm must have terminated with 
a residual graph with no flow augmenting path from t{e) to s{e). Then, as in the 
case where /(e) > c(e), there is a set of nodes, Y with s(e) E Y and t(e) ^ Y 
and it is easy to show that the cocycle condition (f) of Theorem 15.10.12] fails for 

n{Y). 

Therefore, if the algorithm does not fail during every round through the Max-flow algo- 
rithm applied to the modified network, A^, which, as we observed, is the case if condition 
(t) holds, then a channeled flow, /, will be produced and this flow will be a maximum flow. 
This proves the converse of Theorem 15.10.121 

The Max-flow, Min-cut Theorem can also be generalized to channeled flows as follows: 

Theorem 5.10.13 For any network, N = {G,b,c,Vs,Vt), if a flow exists in N, then the 
maximum value, \f\, of any flow, f, in N is equal to the minimum capacity, c{Q{Y)) = 
c{fl'^(Y)) — b{fl^{Y)), of any Vg-Vt-cocycle in N (this means that Vs eY and Vr ^Y). 

If the capacity functions 6 and c have the property that 6(e) < and c(e) > for all 
e G -E, then the condition of Theorem 15.10.121 is trivially satisfied. Furthermore, in this 
case, the flow / = is admissible. Proposition 15.10.61 holds and we can apply directly the 
construction of the residual network, Nf, described above. 



(a) When the Max- flow algorithm terminates, /(e^) = c(er) = 
define / as follows: 

T(^) = / /(e) + /(e,) if e = e 
l/(e) ife^e. 
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A variation of our last problem appears in Gormen, Leiserson, Rivest and Stein [TD] 
(Chapter 26): In this version, the underlying graph, G, of the network, N, is assumed to 
have no parallel edges (and no loops), so that every edge, e, can be identified with the pair, 
{u, v), of its endpoints (so, E (1 V xV). A flow, /, in is a function, / : V xV ^ M., where 
is not necessarily the case that f{u,v) > for all {u,v), but there is a capacity function, 
c: V X V ^ such that c{u, v) > 0, for all {u, v) E V x V and it is required that 

f{v,u) = -fiu,v) and 
f{u,v) < c{u,v), 

for all {u, v) G VxV. Moreover, in view of the skew symmetry condition (/(f , u) = —f{u, v)), 
the equations of conservation of flow are written as 

{u,v)eE 

for all u 7^ Vs, Vt- 

We can reduce this last version of the flow problem to our previous setting by noticing 
that in view of skew symmetry, the capacity conditions are equivalent to having capacity 
functions, 6', and c', defined such that 

h'{u,v) = —c{v,u) 
c{u,v) = c{u,v), 

for every {u, v) E E and / must satisfy 

b'{u,v) < f{u,v) < c'{u,v) 

for all {u,v) G E. However, we must also have f{v,u) = —f{u,v), which is an additional 
constraint in case G has both edges {u,v) and {v,u). This point may be a little confusing 
since in our previous setting, f{u,v) and f{v,u) are independent values. However, this new 
problem is solved essentially as the previous one. The construction of the residual graph is 
identical to the previous case and so is the flow augmentation procedure along an elementary 
path, except that we force fn{v, u) = fniu, v) to hold during this step. For details, the reader 
is referred to Gormen, Leiserson, Rivest and Stein [lOj, Ghapter 26. 

More could be said about flow problems but we believe that we have covered the basics 
satisfactorily and we refer the reader to the various references mentioned in this section for 
more on this topic. 

5.11 Matchings, Coverings, Bipartite Graphs 

In this section, we will be dealing with finite unoriented graphs. Gonsider the following 
problem: We have a set of m machines. Mi, . . . , M^, and n tasks, Ti, . . . , T„. Furthermore, 
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Figure 5.34: A bipartite graph, G, and a maximum matching in G 

each machine, Mj, is capable of performing a subset of tasks. Si C {Ti, . . . ,T„}. Then, the 
problem is to find a set of assignments, {(Mj^, TjJ, . . . , (Mj^^, Tj^)}, with 
{ii, . . . C {1, . . . ,m} and {ji, . . . ,jp} C {1, . . . ,n}, such that 

(1) T,, G^.„ l</.<p; 

(2) p is maximum. 

The problem we just described is called a maximum matching problem. A convenient way 
to describe this problem is to build a graph, G (undirected), with m + n nodes partitioned 
into two subsets X and Y, with X = {xi, . . . , Xm} and Y = {yi, . . . , and with an edge 
between Xi and yj iff Tj G Si, that is, if machine Mj can perform task Tj. Such a graph, G, is 
called a bipartite graph. An example of a bipartite graph is shown in Figure [5. 34[ Now, our 
matching problem is to find an edge set of maximum size, M, such that no two edges share 
a common endpoint or, equivalently, such that every node belongs to at most one edge of 
M. Such a set of edges is called a maximum matching in G. A maximum matching whose 
edges are shown as thicker lines is shown in Figure 15.341 

Definition 5.11.1 A graph, G = {V,E,st), is a bipartite graph iff its set of edges, V, 
can be partitioned into two nonempty disjoint sets, V\,V2, so that for every edge, e E E, 
\st{e) n Vi\ = \st{e) n V2I = 1, i.e., one endpoint of e belongs to Vi while the other belongs 
to V2. 

Note that in a bipartite graph, there are no edges linking nodes in Vi (or nodes in V2). 
Thus, there are no loops. 

Remark: The complete bipartite graph for which |Vi| = m and IV2I = n is the bipartite 
graph that has all edges with i G {1, . . . , m} and j G {1, . . . , n}. This graph is denoted 
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Km,n- The complete bipartite graph ^^3 3 plays a special role, namely, it is not a planar 
graph, which means that it is impossible to draw it on a plane without avoiding that two 
edges (drawn as continuous simple curves) intersect. A picture of K^^s is shown in Figure 
E351 

The maximum matching problem in a bipartite graph can be nicely solved using the 
methods of Section 15.101 for finding Max-flows. Indeed, our matching problem is equivalent 
to finding a maximum flow in the network, N, constructed from the bipartite graph G as 
follows: 

1. Add a new source, Vs and a new sink, Vt] 

2. Add an oriented edge, {vs,u), for every u G Vi, 

3. Add an oriented edge, {v,Vt), for every v G V2; 

4. Orient every edge, e E E, from Vi to V2; 

5. Define the capacity function, c, so that c(e) = 1, for every edge of this new graph. 

The network corresponding to the bipartite graph of Figure 15.341 is shown inFigure 15.361 
Now, it is very easy to check that there is a matching, M , containing p edges iff there is 
a flow of value p. Thus, there is a one-to-one correspondence between maximum matchings 
and maximum integral flows. As we know that the algorithm maxflow (actually, its various 
versions) produces an integral solution when ran on the zero flow, this solution yields a 
maximum matching. 

The notion of graph coloring is also important and has bearing on the notion of bipartite 
graph. 

Definition 5.11.2 Given a graph, G = (V, E, st), a k-coloring of G is a partition of V into 
k pairwise disjoint nonempty subsets, Vi, . . . , V^, so that no two vertices in any subset Vi 
are adjacent (i.e., the endpoints of every edge, e G -E, must belong to Vi and Vj, for some 
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Figure 5.36: The network associated with a bipartite graph 



i ^ j). If a graph, G, admits a fc-coloring, we say that that G is k-colorable. The chromatic 
number, x{Cr), of a graph, G, is the minimum k for which G is /c-colorable. 



Remark: Ahhough the notation, xiG), for the chromatic number of a graph is often used 
in the graph theory hterature, it is an unfortunate choice because it can be confused with 
the Euler characteristic of a graph (see Theorem 15. 12.8I) . Other notations for the chromatic 
number include 7(G), i^{G) and chr(G'). 

The following theorem gives some useful characterizations of bipartite graphs. First, 
we must define the incidence matrix of an unoriented graph, G. Assume that G has edges 
ei, . . . , e„ and vertices vi, . . . , v^. The incidence matrix, A, of G, is the mxn matrix whose 
entries are given by 




1 if Vj e st{ej) 
otherwise. 



Note that, unlike the incidence matrix of a directed graph, the incidence matrix of an 
undirected graph only has non-negative entries. As a consequence, these matrices are not 
necessarily totally unimodular. For example, the reader should check that for any elementary 
cycle, C, of odd length, the incidence matrix. A, of C has a determinant whose value is ±2. 
However, the next theorem will show that the incidence matrix of a bipartite graph is totally 
unimodular and in fact, this property characterizes bipartite graphs. 

In order to prove part of the next theorem we need the notion of distance in a graph, an 
important concept in any case. If G is a connected graph, for any two nodes u and v of G, 
the length of a chain, vr, from u to f is the number of edges in vr and the distance, d{u, v), 
from u to f is the minimum length of all path from u to v. Of course, u = v iS d{u, v) = 0. 

Theorem 5.11.3 Given any graph, G = (y,E,st), the following properties are equivalent: 
(1) G is bipartite. 
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(2) 7(G) = 2. 

(3) G has no elementary cycle of odd length. 

(4) G has no cycle of odd length. 

(5) The incidence matrix of G is totally unimodular. 

Proof . The equivalence (1) <^==^ (2) is clear by definition of the chromatic number. 

(3) <^==^ (4) holds because every cycle is the concatenation of elementary cycles. So, a 
cycle of odd length must contain some elementary cycle of odd length. 

(1) =^ (4). This is because the vertices of a cycle belong alternatively to Vi and V2. So, 
there must be an even number of them. 

(4) =^ (2). Clearly, a graph is fc-colorable iff all its connected components are k- 
colorable, so we may assume that G is connected. Pick any node, fo, in G and let Vi be the 
subset of nodes whose distance from vq is even and V2 be the subset of nodes whose distance 
from t>o is odd. We claim that any two nodes, u and f , in Vi (resp. V2) are not adjacent. 
Otherwise, by going up the chains from u and v back to f and by adding the edge from u 
to f , we would obtain a cycle of odd length, a contradiction. Therefore, G, is 2-colorable. 

(1) =^ (5). Orient the edges of G so that for every e E E, s(e) G Vi and t{e) G V2. 
Then, we know from Proposition 15.8.51 that the incidence matrix, D, of the oriented graph G 
is totally unimodular. However, because G is bipartite, D is obtained from A by multiplying 
all the rows corresponding to nodes in V2 by —1 and so, A is also totally unimodular. 

(5) =^ (3). Let us prove the contrapositive. If G has an elementary cycle, G, of odd 
length, then we observed that the submatrix of A corresponding to G has determinant ±2. 
□ 

We now define the general notion of a matching. 

Definition 5.11.4 Given a graph, G = {V,E,st), a matching, M, in G is a. subset of 
edges so that any two distinct edges in M have no common endpoint (are not adjacent) or 
equivalently, so that every vertex, v & E, is incident to at most one edge in M. A vertex, 
V & V is matched iff it is incident some some edge in M and otherwise it is said to be 
unmatched. A matching, M, is a perfect matching iff every node is matched. 

An example of a perfect matching, M = {{ab), {cd), (e/)}, is shown in Figure 15.371 
with the edges of the matching indicated in thicker lines. The pair {{be), (ed)} is also a 
matching, in fact, a maximal matching (no edge can be added to this matching and still 
have a matching). 

It is possible to characterize maximum matchings in terms of certain types of chains 
called alternating chains defined below: 
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Figure 5.37: A perfect matching in a graph 





Xdtl — 









Figure 5.38: An ahernating chain in G 

Definition 5.11.5 Given a graph, G = {V, E, st), and a matching, M, in G, an elementary 
chain is an alternating chain w.r.t M iff the edges in this chain belong alternately to M and 
E-M. 

Theorem 5.11.6 (Berge) Given any graph, G = {V,E,st), a matching, M, in G is a max- 
imum matching iff there are no alternating chains w.r.t. M whose endpoints are unmatched. 

Proof. First, assume that M is a maximum matching and that G is an alternating chain 
w.r.t. M whose enpoints, u and v are unmatched. An an example, consider the alternating 
chain shown in Figure 15.381 where the edges in C fl M are indicated in thicker lines. 
We can form the set of edges 

M' = {M -{Gn M)) U (C n - M)), 

which consists in deleting the edges in M from G and adding the edges from G not in M. 
It is immediately verified that M' is still a matching but \M'\ = \M\ + 1 (see Figure [5.38p . 
contradicting the fact that M is a maximum matching. Therefore, there are no alternating 
chains w.r.t. M whose endpoints are unmatched. 
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Conversely, assume that G has no alternating chains w.r.t. M whose endpoints are 
unmatched and let M' be another matching with \M'\ > \M\ (i.e., M is not a maximum 
matching). Consider the spanning subgraph, H, of G, whose set of edges is 

(M-M')U(M'-M). 

As M and M' are matchings, the connected components of H are either isolated vertices, 
or elementary cycles of even length, or elementary chains, and in these last two cases, the 
edges in these cycles or chains belong alternately to M and M'; this is because dniu) < 2 
for every vertex u & V and if duiu) = 2, then u is adjacent to one edge in M and one edge 
in M'. 

Now, H must possess a connected component that is a chain, C, whose enpoints are in M', 
as otherwise we would have |M'| < |M|, contradicting the assumption \M'\ > \M\. However, 
G is an alternating chain w.r.t. M whose endpoints are unmatched, a contradiction. □ 

A notion closely related to the concept of a matching but, in some sense, dual, is the 
notion of a line cover. 

Definition 5.11.7 Given any graph, G = {V,E,st), without loops or isolated vertices, a 
line cover (or line covering) of G is a set of edges, C E, so that every vertex u E V is 
incident to some edge in C. A minimum line cover, C, is a line cover of minimum size. 

The maximum matching, M, in the graph of Figure 15.371 is also a minimum line cover. 
The set {{ab), [be), [de), (e/)} is also a line cover but it is not minimum, although minimal. 
The relationship between maximum matchings and minimum covers is given by the following 
theorem: 

Theorem 5.11.8 Given any graph, G = {V,E,st), without loops or isolated vertices, with 
\V\ = n, let M be a maximum matching and let C be a minimum line cover. Then, the 
following properties hold: 

(1) If we associate to every unmatched vertex of V some edge incident to this vertex and 
add all such edges to M , then we obtain a minimum line cover. Cm- 

(2) Every maximum matching, M' , of the spanning subgraph, (V,C), is a maximum match- 
ing of G. 

(3) \M\ + \C\ = n. 

Proof . It is clear that Cm is a line cover. As the number of vertices umatched by M is 
n — 2\M\ (as each edge in M matches exactly two vertices), we have 

\CM\ = \M\+n-2\M\=n-\M\. (*) 
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Furthermore, as C is a minimum line cover, the spanning subgraph, (V,C), does not contain 
any cycle or chain of length greater than or equal to 2. Consequently, each edge e G C — M' 
corresponds to a single vertex unmatched by M'. Thus, 

\C\ - \M'\ = n-2\M'\, 

that is 

\C\=n-\M'\. (**) 

As M is a maximum matching of G, 

\M'\ < \M\ 

and so, using (*) and (**), we get 

\Cm\ = n-\M\ <n - \M'\ = \C\, 
that is, |Cj\/| < \C\. However, C is a minimum matching, so \C\ < \Cm\, which proves that 

|C| = \Cm\- 

The last equation proves the remaining claims. □ 

There are also notions analogous to matchings and line covers but applying to vertices 
instead of edges. 

Definition 5.11.9 Let G = {V, E, st) be any graph. A set, U ^V, of nodes is independent 
(or stable) iff no two nodes in U are adjacent (there is no edge having these nodes as 
endpoints). A maximum independent set is an independent set of maximum size. A set, 
U V, of nodes is a point cover (or vertex cover or transversal) iff every edge of E is 
incident to some node in U. A minimum point cover is a point cover of minimum size. 

For example, {a, b, c, d, /} is point cover of the graph of Figure [5.371 The following simple 
proposition holds: 

Proposition 5.11.10 Let G = {V,E,st) be any graph, U be any independent set, C be any 
line cover, lA be any point cover and M be any matching. Then, we have the following 
inequalities: 

(1) \U\ < \C\; 

(2) \M\ < \U\ 

(3) U is an independent set of nodes iff V — U is a point cover. 
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procedure marking(G', M, mark) 
begin 

for each m G Vi U V2 do mark{u) := endfor; 
while 3m G Vi U V2 with mark{u) = and u not matched by M do 
mark{u) := +; 

while 3v & ViU V2 with mark{v) = and v adjacent to w with mark{w) = + do 
mark{v) := — ; 

if V is not matched by M then exit (a) 
(* an alternating chain has been found *) 
else find w G V^i U V2 so that {vw) G M; mark{w) := + 
endif 
endwhile 
endwhile; 

for each u G Vi with mark{u) = do mark{u) := + endfor; 
for each u E V2 with mark{u) = do mark{u) := — endfor [(3) 
end 



Figure 5.39: Procedure marking 

Proof . (1) Since U is an independent set of nodes, every edge in C is incident with at most 
one vertex in f/, so \U\ < \C\. 

(2) Since M is a matching, every vertex in lA is incident to at most one edge in M, so 
|M| < \U\. 

(3) Clear from the definitions. □ 

It should be noted that the inequalities of Proposition 15.11 .TO] can be strict. For example, 
if G is an elementary cycle with 2/c + 1 edges, the reader should check that both inequalities 
are strict. 

We now go back to bipartite graphs and give an algorithm which, given a bipartite graph, 
G = (Vi U V2,E), will decide whether a matching, M, is a maximum matching in G. This 
algorithm, shown in Figure [5.39[ will mark the nodes with the one of the three tags, +, — , 
or 0. 

The following theorem tells us what is the behavior of the procedure marking. 

Theorem 5.11.11 The procedure marking always terminates in one of the following two 
(mutually exclusive) situations: 

(a) The algorithm finds an alternating chain w.r.t. M whose endpoints are unmatched. 

(b) The algorithm finds a point cover, U, with \U\ = \M\, which shows that M is a maxi- 
mum matching. 
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Proof. Since nodes keep being marked, the algorithm obviously terminates. There are no 
pairs of adjacent nodes bothy marked + since, as soon as a node is marked +, all of its 
adjacent nodes are labeled — . Consequently, if the algorithm ends in [jS), those nodes 
marked — form a point cover. 

We also claim that the endpoints, u and v, of any edge in the matching can't both be 
marked — . Otherwise, by following backward the chains that allowed the marking of u and 
V, we would find an odd cycle, which is impossible in a bipartite graph. Thus, if we end in 
(/?), each node marked — is incident to exactly one edge in M. This shows that the set, U, 
of nodes marked — is a point cover with \U\ = \M\. By Proposition 15.11.101 we see that U 
is a minimum point cover and that M is a maximum matching. 

If the algorithm ends in (a), by tracing the chain starting from the unmatched node, u, 
marked — back to the node marked + causing u to marked, and so on, we find an alternating 
chain w.r.t. M whose endpoints are not matched. □ 

The following important corollaries follow immediately from Theorem 15. 11. lit 

Corollary 5.11.12 In a bipartite graph, the size of a minimum point cover is equal to the 
size of maximum matching. 

Corollary 5.11.13 In a bipartite graph, the size of a maximum independent set is equal to 
the size of a minimum line cover. 

Proof. We know from Proposition 15.11.101 that the complement of a point cover is an in- 
dependent set. Consequently, by Corollary 15.11.12} the size of a maximum independent set 
is n — |M|, where M is a maximum matching and n is the number of vertices in G. Now, 
from Theorem 15.11.81 (3), for any maximum matching, M, and any minimal line cover, C, 
we have |M| + |C| = n and so, the size of a maximum independent set is equal to the size of 
a minimal line cover. □ 

We can derive more classical theorems from the above results. 

Given any graph, G = (V, E, st), for any subset of nodes, U (^V, let 



be the set of neighbours of U, i.e., the set of vertices not in U and adjacent to vertices in U. 

Theorem 5.11.14 (Konig (1931)) For any bipartite graph, G = (Vi U V2,E,st), the maxi- 
mum size of a matching is given by 



Proof . This theorem will follow from Corollary 15.11.121 if we can show that every minimum 
point cover is of the form (V^i — U)U Ng{U), for some subset, U, of Vi. However, a moment 
of reflexion shows that this is indeed the case. □ 

Theorem 15.11.14] implies another classical result: 



Ng{U) = {veV -U\{3ue U){3e e E){st{e) = {u,v})}, 




U\ + \Ng{U)\)- 
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Theorem 5.11.15 (Kdnig-Hall) For any bipartite graph, G = (Vi U V2,E,st), there is a 
matching, M, such that all nodes in V\ are matched iff 

\Ng{U)\ > \U\ for all U C Vi. 

Proof. By Theorem 15 . 11 . 14| there is a matching, M in G with \M\ = \ Vi\ iff 

l^il = min(|Vi -U\ + iNcm) = min(|V^i| + {Nam - |f/|), 

that is, iff \Ng{U) \ -\U\>0 for all U CVi. n 

Now, it is clear that a bipartite graph has a perfect matching (i.e., a matching such that 
every vertex is matched), M, iff |Vi| = IV2I and M matches all nodes in Vi. So, as a corollary 
of Theorem 15 . 1 1 . 15| we see that a bipartite graph has a perfect matching iff |Vi| = IV2I and 
if 

\Ng{U)\>\U\ for all UCVi. 
As an exercise, the reader should show the 

Marriage Theorem (Hall, 1935) Every k-regular bipartite graph, with k > 1, has a perfect 
matching (a graph is fc-regular iff every node has degree k). 

For more on bipartite graphs, matchings, covers, etc., the reader should consult Diestel 
p!3] (Chapter 2), Berge [3] (Chapter 7) and also Harary [28] and BoUobas [7]. 



5.12 Planar Graphs 

Suppose we have a graph, G, and that we want to draw it "nicely" on a piece of paper, which 
means that we draw the vertices as points and the edges as line segments joining some of 
these points, in such a way that no two edges cross each other, except possibly at common 
endpoints. We will have more flexibility and still have a nice picture if we allow each abstract 
edge to be represented by a continuous simple curve (a curve that has no self- intersection), 
that is, a subset of the plane homeomorphic to the closed interval [0, 1] (in the case of a loop, 
a subset homeomorphic to the circle, S^). If a graph can be drawn in such a fashion, it is 
called a planar graph. For example, consider the graph depicted in Figure I5.40[ 

If we look at Figure 15.401 we may believe that the graph G is not planar, but this is no 
so. In fact, by moving the vertices in the plane and by continuously deforming some of the 
edges, we can obtain a planar drawing of the same graph, as shown in Figure 15.411 

However, we should not be overly optimistic. Indeed, if we add an edge from node 5 to 
node 4, obtaining the graph known as shown in Figure [5.421 it can be proved that there is 
no way to move the nodes around and deform the edge continuously to obtain a planar graph 
(we will prove this a little later using the Euler formula). Another graph that is non-planar 
is the bipartite grapk -^"3,3. The two graphs, and -ft'3^3 play a special role with respect to 
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Figure 5.42: The complete graph K^, a non-planar graph 
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planarity. Indeed, a famous theorem of Kuratowski says that a graph is planar if and only 
if it does not contain or ^3^3 as a minor (we will explain later what a minor is). 

Remark: Given n vertices, say {1, . . . , n}, the graph whose edges are all subsets {i,j}, with 
i,j e {1, . . . ,n} and i ^ j, is the complete graph on n vertices and is denoted by Kn (but 
Diestel uses the notation K'^). 

In order to give a precise definition of a planar graph, let us review quickly some basic 
notions about curves. A simple curve (or Jordan curve) is any injective continuous function, 
7: [0, 1] M^. Since [0, 1] is compact and 7 is continuous, it is well-known that the inverse, 
f~^: 7([0, 1]) [0, 1], of / is also continuous. So, 7 is a homeomorphism between [0, 1] and 
its image, 7([0, 1]). With a slight abuse of language we will also call the image, 7([0, 1]), 
of 7, a simple curve. This image is a connected and compact subset of R^. The points 
a — 7(0) and h — 7(1) are called the boundaries or endpoints of 7 (and 7([0, 1])). The 
open subset 7([0, 1]) — {7(0), 7(1)} is called the interior of 7([0, 1]) and is denoted 7. A 
continuous function , 7: [0,1] M^, such that 7(0) = 7(1) and 7 is injective on [0,1) 
is called a simple closed curve or simple loop or closed Jordan curve. Again, by abuse 
of language, we call the image, 7([0, 1]), of 7, a simple closed curve etc. Equivalently, if 

— {{x,y) e \ -\- — 1} is the unit circle in R^, a simple closed curve is any subset 
of R^ homeomorphic to . In this case, we call 7(0) = 7(1) the boundary or base point of 
7. The open subset 7([0, 1]) — {7(0)} is called the interior of 7([0, 1]) and is also denoted 7. 

Remcirk: The notions of simple curve and simple closed curve also make sense if we replace 
R^ by any topological space, X, in particular, a surface (In this case, a simple (closed) curve 
is a continuous injective function 7: [0, 1] — > X etc.). 

We can now define plane graphs as follows: 

Definition 5.12.1 A plane graph is a pair, Q = {V, E), where V is a finite set of points in 
R^, £■ is a finite set of simple curves and closed simple curves in R^ called edges and loops, 
respectively, and satisfying the following properties: 

(i) The endpoints of every edge in E are vertices in V and the base point of every loop is 
a vertex in V. 

(ii) The interior of every edge contains no vertex and the interiors of any two distinct 
edges are disjoint. Equivalently, every edge contains no vertex except for its boundaries 
(base point in the case of a loop) and any two distinct edges intersect only at common 
boundary points. 

We say that G is a simple plane graph if it has no loops and if different edges have different 
sets of endpoints 

Obviously, a plane graph, Q — {V,E), defines an "abstract graph", G — {V,E,st), such 
that 
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(a) For every simple curve, 7, 

st(7) = {7(0),7(1)} 

(b) For every simple closed curve, 7, 

st{^) = {7(0)}. 

For simplicity of notation, we will usually write Q for both the plane graph and the 
abstract graph associated with Q. 

Definition 5.12.2 Given an abstract graph, G, we say that G is a planar graph iff there 
is some plane graph, Q, and an isomorphism, ip: G Q, between G and the abstract graph 
associated with Q. We call ip an embedding of G in the plane or a planar embedding of G. 



Remarks: 

1. If G is a simple planar graph, then by a theorem of Fary, G can be drawn as a plane 
graph in such a way that the edges are straight line segments (see Gross and Tucker 
[25], Section 1.6). 

2. In view of the remark just before Definition 15. 12. 1[ given any topological space, X, for 
instance, a surface, we can define a graph on X as a pair, (V^, E), where is a finite 
set of points in X and E is a. finite sets of simple (closed) curves on X satisfying the 
conditions of Definition 15.12.11 

3. Recall the stereographic projection (from the north pole), a^: [S'^ — {N}) —>■ M^, from 
the sphere, S"^ = {{x,y,x) E M.^ \ x"^ + y'^ + z"^ = 1} onto the equatorial plane, z = 0, 
with N = (0, 0, 1) (the north pole), given by 

aN{x, y,z) = ( — ) . 

\1 — z I — z J 

We know that ctat is a homeomorphism, so if 99 is a planar embedding of a graph G into 
the plane, then cr^^ o is an embedding of G into the sphere. Conversely, if ip is an 
embedding of G into the sphere, then o is a planar embedding of G. Therefore, 
a graph can be embedded in the plane iff it can be embedded in the sphere. One of 
the nice features of embedding in the sphere is that the sphere is compact (closed and 
bounded), so the faces (see below) of a graph embedded in the sphere are all bounded. 

4. The ability to embed a graph in a surface other that the sphere broadens the class of 
graphs that can be drawn without pairs of intersecting edges (except at endpoints). 
For example, it is possible to embed and K^^^ (which are known not to be planar) 
into a torus (try it!). It can be shown that for every (finite) graph, G, there is some 
surface, X, such that G can be embedded in X. Intuitively, whenever two edges cross 
on a sphere, by lifting one of the two edges a little bit and adding a "handle" on which 
the lifted edge lies we can avoid the crossing. An excellent reference on the topic of 
graphs on surfaces is Gross and Tucker [25j. 
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One of the new ingredients of plane graphs is that the notion of a face makes sense. 
Given any nonempty open subset, Q, of the plane M?, we say that two points, a,b & Q are 
(arcwise) connected^ iff there is a simple curve, 7, such that 7(0) = a and 7(1) = h. Being 
connected is an equivalence relation and the equivalence classes of VL w.r.t. connectivity are 
called the connected components (or regions) of VL. Each region is maximally connected and 
open. If R is any region of VL and if we denote the closure of R (i.e., the smallest closed set 
containing R) by i?, then the set dR = R — R is also a closed set called the boundary (or 
frontier) of R. 

Now, given a plane graph, Q, if we let \Q\ be the the subset of consisting of the union 
of all the vertices and edges of Q, then this is a closed set and it complement, f2 = — |^|, 
is an open subset of M^. 

Definition 5.12.3 Given any plane graph, Q, the regions of f2 = — \Q\ are called the 
faces of Q. 

As expected, for every face, F, of Q, the boundary, dF, of F is the subset, \H\, associated 
with some subgraph, Ti, of Q. However, one should observe that the boundary of a face may 
be disconnected and may have several "holes" . The reader should draw lots of planar graphs 
to understand this phenomenon. Also, since we are considering finite graphs, the set \Q\ is 
bounded and thus, every plane graph has exactly one unbounded face. Figure 15.431 shows 
a planar graph and its faces. Observe that there are five faces, where A is bounded by the 
entire graph, B is bounded by the triangle (4, 5, 6) the outside face, C, is bounded by the 
two edges from 8 to 2, the loop around node 2, the two edges from 2 to 7 and the outer edge 
from 7 to 8, D is bounded by the two edges between 7 and 8, and E is bounded by the loop 
around node 2. 

Remarks: 

1. Using (inverse) stereographic projection, we see that all the faces of a graph embedded 
in the sphere are bounded. 

2. If a graph, G, is embedded in a surface, S, then the notion of face still makes sense. 
Indeed, the faces of G are the regions of the open set = S" — IGj. 

Actually, one should be careful (as usual) not to rely too much on intuition when dealing 
with planar graphs. Although certain facts seem obvious, they may turn out to be false after 
closer scrutiny and when they are true, they may be quite hard to prove. One of the best 
examples of an "obvious" statement whose proof is much less trivial than one might expect 
is the Jordan curve theorem which is actually needed to justify certain "obvious" facts about 
faces of plane graphs. 

"^In topology, a space is connected iff it cannot be expressed as the union of two nonempty disjoint open 
subsets. For open subsets of R", connectedness is equivalent to arc connectedness. So it is legitimate to use 
the term connected. 
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Figure 5.43: A planar graph and its faces 

Theorem 5.12.4 (Jordan Curve Theorem) Given any closed simple curve, 7, in M, the 
complement, — 7([0, 1]), o/7([0, 1]), consist of exactly two regions both having 7([0, 1]) as 
boundary. 

Proof . There are several proofs all using machinery (such as homology or differential topol- 
ogy) beyond the scope of these notes. A proof using the notion of winding number is given 
in Guillemin and Pollack [20] (Chapter 2, Section 5) and another proof using homology can 
be found in Munkres [33] (Chapter 4, Section 36). □ 

Using Theorem 15.12.41 the following properties can be proved: 

Proposition 5.12.5 Let Q = {V,E) be any plane graph and let e ^ E be any edge of Q . 
Then the following properties hold: 

(1) For any face, F of Q , either e df or dF fl e = 0. 

(2) If e lies on a cycle, C, of Q , then e lies on the boundary of exactly two faces of G and 
these are contained in distinct faces of C. 

(3) If e lies on no cycle, then e lies on the boundary of exactly one face of Q . 

Proof . See Diestel |13J, Section 4.2. 
As a corollaries, we also have 

Proposition 5.12.6 Let Q = {V,E) be any plane graph and let F be any face of Q. Then, 
the boundary, dF, of F is a subgraph of Q (more accurately, d = \H\, for some subgraph, Ti, 
ofG). 
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Proposition 5.12.7 Every plane forest has a single face. 

One of the main theorems about plane graphs is the so-called Euler formula. 

Theorem 5.12.8 Let G be any connected planar graph with uq vertices, ui edges and n2 
faces. Then, we have 

Uq- ni+n2 = 2. 

Proof . We proceed by induction on ni. If ni = 0, the formula is trivially true, as uq = n2 = 1. 
Assume the theorem holds for any ui < n and let G be a connected planar graph with n 
edges. If G has no cycle, then as it is connected, it is a tree, no = n + 1 and n2 = 1, so 
nQ — ni+n2 = n+l — n+1 = 2, as desired. Otherwise, let e be some edge of G belonging to a 
cycle. Consider the graph G' = (y,E — {e}), it is still a connected planar graph. Therefore, 
by the induction hypothesis, 

% ^ ('^i ^ 1) + ^2 — 2- 

However, by Proposition 15.12.51 as e lies on exactly two faces of G, we deduce that 
n2 = n'2 + 1. Consequently 

2 = no - (ni - 1) + ^2 = no - ni + 1 + - 1 = no - n-i + n2, 

establishing the induction hypothesis. □ 

Remarks: 

1. Euler 's formula was already known to Descartes in 1640 but the first proof by given 
by Euler in 1752. Poincare generalized it to higher-dimensional polytopes. 

2. The numbers no, ni, and n2 are often denoted by n^, and Uf {v for vertex, e for 
edge and / for face). 

3. The quantity no — ni + n2 is called the Euler characteristic of the graph G and it is 
usually denoted by xg- 

4. If a connected graph, G, is embedded in a surface (orientable), S, then we still have 
an Euler formula of the form 

no - ni + n2 = x{^) ='2-'2g, 

where x{S) is a number depending only on the surface, S, called the Euler characteristic 
of the surface and g is called the genus of the surface. It turns out that g > is 
the number of "handles" that need to be glued to the surface of a sphere to get a 
homeomorphic copy of the surface S. For on this fascinating subject, see Gross and 
Tucker [25j. 
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It is really remarkable that the quantity Uq — ni+ n2 is independent of the way a planar 
graph is drawn on a sphere (or in the plane). A neat application of Euler's formula is the 
proof that there are only five regular convex polyhedra (the so-called platonic solids). Such 
a proof can be found in many places, for instance Berger [4J and Cromwell [11]. It is easy to 
generalize Euler's formula to planar graphs that are not necessarily connected. 

Theorem 5.12.9 Let G be any planar graph with uq vertices, ni edges, n2 faces and c 
connected components. Then, we have 

riQ - rii + n2 = c + 1. 

Proof. Reduce the proof of Theorem 15 . 1 2 . 91 to the proof of Theorem 15 . 1 2 . 8 1 by adding vertices 
and edges between connected components to make G connected. Details are left as an 
exercise. □ 

Using the Euler formula we can now prove rigorously that and K^^s are not planar 
graphs. For this, we will need the following fact: 

Proposition 5.12.10 If G is any simple, connected, plane graph with ni > 3 edges and n2 
faces, then 

2ni > 3n2. 

Proof. Let F{G) be the set of faces of G. Since G is connected, by Proposition 15.12.5) (2), 
every edge belongs to exactly two faces. Thus, if sj? is the number of sides of a face, F, of 
G, we have 

Sf = 2ni. 

FeF(G) 

Furthermore, as G has no loops, no parallel edges and uq > 3, every face has at least three 
sides, i.e., > 3. It follows that 

2ni = Sf > 3n2, 

FeF(G) 

as claimed. □ 

The proof of Proposition 15.12.101 shows that the crucial constant on the right-hand the 
inequality is the the minimum length of all cycles in G. This number is called the girth of 
the graph G. The girth of a graph with a loop is 1 and the girth of a graph with parallel 
edges is 2. The girth of a tree is undefined (or infinite). Therefore, we actually proved: 

Proposition 5.12.11 If G is any connected, plane graph with ni edges and n2 faces and G 
is not a tree, then 

2ni > girth(G') n2. 
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Corollary 5.12.12 If G is any simple, connected, plane graph with n > 3 nodes then G has 
at most 3r2 — 6 edges and 2n — A faces. 

Proof. By Proposition 15.12. 10[ we have 2ni > 3n2, where ni is the number of edges and n2 
is the number of faces. So, n2 < |ni and by Euler's formula 

n - ni + ^2 = 2, 

we get 

2 

n- ni + -Til > 2, 

that is, 

1 

n ni > 2, 

3 ' 

namely ni < 3n — 6. Using n2 < ^ni, we get ?t,2 < 2r;, — 4. □ 
Corollary 5.12.13 The graphs and K^^^ are not planar. 

Proof . We proceed by contradiction. First, consider K^,. We have uq = 5 and has rii = 10 
edges. On the other hand, by Corollary 15 . 1 2 . 1 21 should have at most 3x5 — 6=15 — 6 = 9 
edges, which is absurd. 

Next, consider K^ ^. We have uq = Q and ^^3^3 has ni = 9 edges. By the Euler formula, 
we should have 

^2 = 9-6 + 2 = 5. 

Now, as -^"3^3 is bipartite, it does not contain any cycle of odd length, and so each face has 
at least four sides, which implies that 

2ni > 4n2 

(because the girth of K^^^ is 4.) So, we should have 

18 = 2 ■ 9 > 4 ■ 5 = 20, 

which is absurd. □ 

Another important property of simple planar graph is the following: 

Proposition 5.12.14 If G is any simple, planar graph, then there is a vertex, u, such that 
dciu) < 5. 

Proof. If the property holds for any connected component of G, then it holds for G, so we 
may assume that G is connected. We already know from Proposition 15. 12.T01 that 2ni > 3^2. 
i.e. 

2 

^2 < 2^1- (*) 
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If dciu) > 6 for every vertex, u, as '^Xu^vdci'^) = 2ni, then Guq < 2ni, i.e., Uq < rii/S. By 
Euler's formula, we would have 

1 2 

n2 = ni - no + 2 > 111 - -ni + 2 > -ni, 

o o 

contradicting (*). □ 

Remarkably, Proposition 15.12.141 is the key ingredient in the proof that every planar 
graph is 5- colorable. 

Theorem 5.12.15 Every planar graph, G, is 5-colorable. 

Proof. Clearly, parallel edges and loop play no role in finding a coloring of the vertices of G, 
so we may assume that G is a simple graph. Also, the property is clear for graphs with less 
than 5 vertices. We will proceed by induction on the number of vertices, m. By Proposition 
15.12.141 the graph G has some vertex, uq, with dc{u) < 5. By the induction hypothesis, we 
can color the subgraph, G' , induced by V — {uq} with 5 colors. If d{uo) < 5, we can color 
Uq with one of the colors not used to color the nodes adjacent to Uq (at most 4) and we are 
done. So, assume dc{uo) = 5 and let f i, . . . , fs be the nodes adjacent to Uq and encountered 
in this order when we rotate counter-clockwise around Uq (see Figure 15.441) . If fi, . . . , ^5 are 
not colored with different colors, again, we are done. 

Otherwise, by the induction hypothesis, let {Xi,...,X^} be a coloring of G' and, by 
renaming the Xj's if necessary, assume that f « G Xj, for i = 1, . . . , 5. There are two cases: 

(1) There is no chain from Vi to ^3 whose nodes belong alternately to Xi and X2. If 
so, f 1 and V3 must belong to different connected components of the subgraph, H', of 
G' induced by Xi U X2. Then, we can permute the colors 1 and 3 in the connected 
component of H' that contains V3 and color uq with color 3. 

(2) There is a chain from vi to ^3 whose nodes belong alternately to Xi and X2. In this 
case, as G is a planar graph, there can't be any chain from V2 to V4 whose nodes 
belong alternately to X2 and X4. So, V2 and W4 do not belong to the same connected 
component of the subgraph, H", of G' induced by X2 UX4. But then, we can permute 
the colors 2 and 4 in the connected component of H" that contains V4 and color uq 
with color 4. □ 

Theorem 15.12.151 raises a very famous problem known as the four color problem: Can 
every planar graph be colored with four colors? 

This question was apparently first raised by Francis Guthrie in 1850, communicated to 
De Morgan by Guthrie's brother Frederick in 1852 and brought to the attention to a wider 
public by Cayley in 1878. In the next hunded years, several incorrect proofs were proposed 
and this problem became know as the four color conjecture. Finally, in 1977, Appel and 
Haken gave the first "proof" of the four color conjecture. However, this proof was somewhat 
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Figure 5.44: The 5 nodes adjacent to uq 

controversial for various reasons, one of the reasons being that it rehes on a computer program 
for checking a large number of unavoidable configurations. Appel and Haken subsequently 
published a 741 page paper correcting a number of errors and addressing various criticisms. 
More recently (1997) a much shorter proof, still relying on a computer program, but a lot 
easier to check (including the computer part of it) has been given by Robertson, Sanders, 
Seymour and Thomas. For more on the four color problem, see Diestel [13], Chapter 5, and 
the references given there. 

let us now go back to Kuratowski's criterion for non-planarity. For this, it is useful to 
introduce the notion of edge contraction in a graph. 

Definition 5.12.16 Let G = {V, E, st) be any graph and let e be any edge of G. The graph 
obtained by contracting the edge e into a new vertex, Ve, is the graph, G/e = {V',E',st'), 
with V = {V — st{e)) U {Vf.} where Ve is a new node [ve ^ V"); E' = E — {e}; and with 

if st{e') n st{e) = 
if st(e') = st{e) 

if st{e') n st{e) = {z} and st(e') = {u, z} with z 
if st(e') = {x} or st(e') = {?/} with st{e) = {x,y}. 

If G is a simple graph, then we need to eliminate parallel edges and loops. In, this case, 
e = {x,y} and G/e = (y',E',st) is defined so that V = {V — {x,y}) U {ve} where Ve is a 
new node and 

E' = {{u,v} I {u,v}n{x,y} = ^ 

{{u,Ve}\ {u,x} e E - {e} or {u,y} e E - {e}}. 

Fi gure [5.451 shows the result of contracting the upper edge {2, 4} (shown as a thicker line) 
in the graph shown on the left, which is not a simple graph. Observe how the lower edge 
{2,4} becomes a loop around 7 and the two edges {5,2} and {5,4} become parallel edges 
between 5 and 7. 



st'i 
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Figure 5.45: Edge Contraction in a graph 




Figure 5.46: Edge Contraction in a simple graph 



Figure 15.461 shows the result of contracting edge {2, 4} (shown as a thicker line) in the 
simple graph shown on the left. This time, the two edges {5, 2} and {5,4} become a single 
edge and there is no loop around 7 as the contracted edge is deleted. 

Now, given a graph, G, we can repeatedly contract edges. We can also take a subgraph 
of a graph G and then perform some edge contractions. We obtain what is known as a minor 
of G. 

Definition 5.12.17 Given any graph, G, a graph, H, is a minor of G is there is a sequence 
of graphs, Hq, Hi, . . . , Hn {n > 1), such that 

(1) Hq = G; Hn = H] 

(2) Either ilj+i is obtained from H^ by deleting some edge or some node of H^ and all the 
edges incident with this node, or 

(3) Hi^i is obtained from H^ by edge contraction, 

with 0<i<n — 1. KG is a simple graph, we require that edge contractions be of the 
second type described in Definition 15.12.161 so that if is a a simple graph. 

It is easily shown that the minor relation is a partial order on graphs (and simple graphs). 
Now, the following remarkable theorem originally due to Kuratowski characterizes planarity 
in terms of the notion of minor: 
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Theorem 5.12.18 (Kuratowski, 1930) For any graph, G, the following assertions are equiv- 
alent: 

(1) G is planar; 

(2) G contains neither nor K^^^ as a minor. 

Proof . The proof is quite involved. The first step is to prove the theorem for 3-connected 
graphs. (A graph G = (y,E) is /c-connected iff |V"| > k and iff every graph obtained by 
deleting any set, S V, of nodes with |S'| < A; and the edges incident to these node is 
still connected. So, a 1-connected graph is just a connected graph.) We refer the reader to 
Diestel [13], Section 4.4, for a complete proof. □ 

Another way to state Kuratowski's theorem involves edge subdivision, an operation of 
independent interest. Given a graph, G = (V, E, st), possibly with loops and parallel edges, 
the result of subdividing an edge, e, consists in creating a new vertex, Ve, deleting the edge 
e, and adding two new edges from Ve to the old endpoints of e (possibly the same point). 
Formally, we have the following definition: 

Definition 5.12.19 Given any graph, G = {V,E,st), for any edge, e & E, the result of 
subdividing the edge e is the graph, G' = (V U {fe}, {E — {e}) U {e\ e^}, st'), where Ve is a 
new vertex and e^, are new edges, st'{e') = st{e') for all e' E E — {e} and if st{e) = {u, v} 
{u = V is possible), then st'(e^) = {ve,u} and st'(e^) = {ve,v}. If a graph, G', is obtained 
from a graph, G, by a sequence of edge subdivisions, we say that G' is a subdivision of G. 

Observe that by repeatedly subdividing edges, any graph can be transformed into a 
simple graph. Given two graphs, G and H, we say that G and H are homeomorphic iff 
they have respective subdivisions G' and H' that are isomorphic graphs. The idea is that 
homeomorphic graphs "look the same" , viewed as topological spaces. Figure 15.471 shows an 
example of two homeomorphic graphs. A graph, H, that has a subdivision, H', which is 
a subgraph of some graph, G, is called a topological minor of G. Then, it is not hard to 
show (see Diestel [I3], Chapter 4, or Gross and Tucker [25], Chapter 1) that Kuratowski's 
Theorem is equivalent to the statement 

A graph, G, is planar iff it does not contain any subgraph homeomorphic to either or 
3 or, equivalently, if it has has neither nor K^ ^ as a topological minor. 

Another somewhat surprising characterization of planarity involving the concept of cycle 
space over F2 (see Definition 15.7.101 and the Remarks after Theorem 15.7.161) and due to 
MacLane is the following: 

Theorem 5.12.20 (MacLane, 1937) A graph, G is planar iff its cycle space, T , over F2 
has a basis such that every edge of G belongs to at most two cycles of this basis. 
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Figure 5.47: Two homeomorphic graphs 



Proof. See Diestel [13], Section 4.4. □ 

We conclude this section on planarity with a brief discussion of the dual graph of a 
plane graph, a notion originally due to Poincare. Duality can be generalized to simplicial 
complexes and relates Voronoi diagrams and Delaunay triangulations, two very important 
tools in computational geometry. 

Given a plane graph, G = {V,E), let F{G) be the set of faces of G. The crucial point 
is that every edge of G is part of the boundary of at most two faces. A dual graph, G* = 
{V*, E*), of G is a graph whose nodes are in one-to-one correspondence with the faces of G, 
whose faces are in one-to-one correspondence with the nodes of G and whose edges are also 
are in one-to-one correspondence with the the egdes of G. For any edge, e E E, a. dual edge, 
e*, links the two nodes and vp^ associated with the faces Fi and F2 adjacent to e or, e* 
is a loop from vp to itself if e is ajacent to a single face. Here is the precise definition: 

Definition 5.12.21 Let G = {V,E) be a plane graph and let F{G) be its set of faces. A 
dual graph of G is a graph, G* = (V*, E*), where 

(1) V* = {vp I F G F{G)}, where vp is a point chosen in the (open) face, F, of G; 

(2) E* = {e* I e G E}, where e* is a simple curve from vp-^ to vp^ crossing e, if e is part of 
the boundary of two faces Fi and F2 or else, a closed simple curve crossing e from vp 
to itself, if e is part of the boundary of exactly one face, F. 

Q o 

(3) For each e E E, we have e*nG = enG*=e fl e*,a one point set. 

An example of a dual graph is shown in Figure 15.481 The graph G has four faces, a, b, c, d 
and the dual graph, G*, has nodes also denoted a, b, c, d enclosed in a small circle, with the 
edges of the dual graph shown with thicker lines. 

Note how the edge {5, 6} gives rise to the loop from d to itself and that there are parallel 
edges between d and a and between d and c. Thus, even if we start with a simple graph, a 
dual graph may have loops and parallel edges. 

Actually, it is not entirely obvious that a dual of a plane graph is a plane graph but 
this is not difficult to prove. It is also important to note that a given plane graph, G, does 
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Figure 5.48: A graph and its dual graph 

not have a unique dual since the vertices and the edges of a dual graph can be chosen in 
infinitely different ways in order to satisfy the conditions of Definition I5.12.21[ However, 
given a plane graph, G, if Hi and H2 are two dual graphs of G, then it is easy to see that Hi 
and H2 are isomorphic. Therefore, with a slight abuse of language, we may refer to "the" 
dual graph of a plane graph. Also observe that even if G is not connected, its dual, G*, is 
always connected. 

The notion of dual graph applies to a plane graph and not to a planar graph. Indeed, 
the graphs G* and G2 associated to two different embeddings, Gi and G2, of the same 
abstract planar graph, G, may not be isomorphic, even though Gi and G2 are isomorphic 
as abstact graphs. For example, the two plane graphs, Gi and G2, shown in Figure EIH] are 
isomorphic but their dual graphs, G* and G2, are not, as the reader should check (one of 
these two graphs has a node of degree 7 but for the other graph all nodes have degree at 
most 6). 

Remark: If a graph, G, is embedded in a surface, S, then the notion of dual graph also 
makes sense. More for on this, see Gross and Tucker |25j . 

In the following proposition, we summarize some useful properties of dual graphs. 
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Figure 5.49: Two isomorphic plane graphs whose dual graphs are not isomorphic 




Figure 5.50: A self-dual graph 

Proposition 5.12.22 The dual, G* of any plane graph is connected. Furthermore, if G is 
a connected plane graph, then G** is isomorphic to G. 

Proof. Left as an exercise. 

We a slight abuse of notation we often write G** = G (when G is connected). A plane 
graph, G, whose dual, G*, is equal to G (i.e., isomorphic to G) is called self-dual. For 
example, the plane graph shown in Figure 15.501 (the projection of a tetrahedron on the 
plane) is self dual. 

The duality of plane graphs is also reflected algebraically as a duahty between their cycle 
spaces and their cut spaces (over F2). 

Proposition 5.12.23 If G is any connected plane graph, G, then the following properties 
hold: 

(1) A set of edges, C ^ E, is a cycle in G iff C* = {e* G i?* | e G C} is a minimal cutset 
m G*. 

(2) If J-'{G) and T{G*) denote the cycle space of G over ¥2 and the cut space of G* over 
¥2, respectively, then the dual, J^*{G), of J^{G) (as a vector space) is equal to the cut 
space, T{G*), of G* , i.e., 

J^*{G) = T{G*). 
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(3) IfT is any spanning tree ofG, then (V*, {E ~ E{T))*) is a spanning tree of G* (Here, 
E(T) is the set of edges of the tree, T .) 

Proof . See Diestel [13j, Section 4.6. □ 

The interesting problem of finding an algorithmic test for planarity has received quite a 
bit of attention. Hopcroft and Tarjan have given an algorithm running in linear time in the 
number of vertices. More more on planarity, the reader should consult Diestel [13j, Chapter 
4, or Harary [28j, Chapter 11. 

Besides the four color "conjecture", the other most famous theorem of graph theory is 
the graph minor theorem, due to Roberston and Seymour and we can't resist stating this 
beautiful and amazing result. For this, we need to explain what is a well-quasi order, for 
short, a w.q.o. 

Recall that a partial order on a set, X, is a binary relation, <, which is reflexive, sym- 
metric and anti-symmetric. A quasi-order (or preorder) is a relation which is reflexive and 
transitive (but not necessarily anti-symmetric). A w ell- quasi- order , for short, a w.g.o , is a 
quasi-order with the following property: 

For every infinite sequence, (xn)„>i, of elements Xi G X, there exist some indices, j, 
with 1 < z < j, so that Xi < Xj. 

Now, we know that being a minor of another graph is a partial order and thus, a quasi- 
order. Here is Robertson and Seymour's theorem: 

Theorem 5.12.24 ( Graph Minor Theorem, Robertson and Seymour, 1985-2004 ) The minor 
relation on finite graphs is a well quasi-order. 

Remarkably, the proof of Theorem 15.12.241 is spread over 20 Journal papers (under the 
common title. Graph Minors) written over nearly 18 years and taking well over 500 pages! 
Many original techniques had to be invented to come up with this proof, one of which is a 
careful study of the conditions under which a graph can be embedded in a surface and a 
"Kuratowski-type" criterion based on a finite family of "forbidden graphs" . The interested 
reader is urged to consult Chapter 12 of Diestel [13j and the references given there. 

A precursor of the graph minor theorem is a theorem of Kruskal (1960) which applies to 
trees. Although much easier to prove that the graph minor theorem, the proof fo Kruskal's 
Theorem is very ingenious. It turns out that there are also some interesting connections 
between Kruskal's Theorem and proof theory, due to Harvey Friedman. A survey on this 
topic can be found in Gallier |15j . 
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